Processing / tapping an HLS audio stream (or global app output)

Media Technologies Audio
c0delogic OP
Created Apr ’16
Replies 9
Boosts 0
Views 4.5k
Participants 10

I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). It doesn't seem like attaching an AVAudioMix configured with with an `audioTapProcessor` has any effect; none of the callbacks except `init` are being invoked. Is this a known limitation? If so, is this documented somewhere?


If the above is a limitation, what are my options using some of the other audio APIs? I looked into `AVAudioEngine` as well but it doesn't seem like there's any way I can configure any of the input node types to use an HLS stream. Am I wrong? Are there lower level APIs available to play HLS streams that provide the necessary hooks?


Alternatively, is there some generic way to tap into all audio being output by my app regardless of its source?


Thanks a lot!

Answered by theanalogkid in 139608022

The MTAudioProcessingTap is not available with HTTP live streaming. I suggest filing an enhancement if this feature is important to you - and it's usually helpful to describe the type of app you're trying to design and how this feature would be used.

Replies  9
Boosts  0
Views  4.5k
Participants  10
JacobSyndeo OP
May ’16

Bumping this & jumping on the notifications, as I too would really like to know if this is possible.

0
theanalogkid OP
May ’16
Accepted Answer

The MTAudioProcessingTap is not available with HTTP live streaming. I suggest filing an enhancement if this feature is important to you - and it's usually helpful to describe the type of app you're trying to design and how this feature would be used.

1
gaudiolab OP
Oct ’18

Hi theanalogkid,We are developing VR video player and recently we want to support spatial audio. Our player is based on AVPlayer to play HLS video (m3u8), which is 360 video with Ambisonics (https://en.wikipedia.org/wiki/Ambisonics). To process Ambisonics, we need a path to get PCM audio (MTAudioProcessingTAP) for hls stream, but it is not available as c0delogic reported. So is there plan to update this?

0
JUN1OKAMURA OP
Jul ’20
Hi, I am also working AVPlayer to play HLS audio (m3u8), and put 3D surround processing as like as gaudiolab, how can I put MTAudioProcessingTAP for HLS stream and let me know updates.
0
bedrock OP
Jul ’20
Same here.

We are developing an AR application and we want to capture the AR experience (HLS video with audio attached to an object).

As a hotfix we are capturing the audio over the speakers via the microphone and mixing it with a screen capture afterwards into one media file. Not very satisfying in terms of UX, quality and performance.
0
Eric_WVGG OP
Mar ’21
I am attempting to do visualizations (audio volume, visualized pitch and the like) in real time, and could use this ability.
0
austino222 OP
Mar ’21
Same here. We're trying to do simple capture of LPCM sample flowing into the hardware.
0
peurocs4 OP
Sep ’23

Any news??

0
tnaruto OP
1w

We have got no update yet?

0
Processing / tapping an HLS audio stream (or global app output)
First post date Last post date
 
 
Q