Convert a CMSampleBuffer to AVAudioPCMBuffer

Question

Created Oct ’19

Replies 3

Boosts 0

Views 7.2k

Participants 4

I'm trying to convert a CMSampleBuffer to a AVAudioPCMBuffer instance to be able to perform audio processing in realtime. I wrote an optional initialiser for my extension to pass a CMSampleBuffer reference. Unfortunately I simply don't know how to write to the AVAudioPCMBuffer's data. Here is my code so far:

import AVKit

extension AVAudioPCMBuffer {
    static func create(from sampleBuffer: CMSampleBuffer) -> AVAudioPCMBuffer? {
        
        guard let description: CMFormatDescription = CMSampleBufferGetFormatDescription(sampleBuffer),
            let sampleRate: Float64 = description.audioStreamBasicDescription?.mSampleRate,
            let numberOfChannels: Int = description.audioChannelLayout?.numberOfChannels
            else { return nil }
        
        guard let blockBuffer: CMBlockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) else {
            return nil
        }

        let length: Int = CMBlockBufferGetDataLength(blockBuffer)
        
        let audioFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: sampleRate, channels: AVAudioChannelCount(numberOfChannels), interleaved: false)
        let buffer = AVAudioPCMBuffer(pcmFormat: audioFormat!, frameCapacity: AVAudioFrameCount(length))!
        buffer.frameLength = buffer.frameCapacity
        
        for channelIndex in 0...numberOfChannels - 1 {
            guard let channel: UnsafeMutablePointer<Float> = buffer.floatChannelData?[channelIndex] else { return nil }
            
            for pointerIndex in 0...length - 1 {
                let pointer: UnsafeMutablePointer<Float> = channel.advanced(by: pointerIndex)
                pointer.pointee = 100
            }
        }
        
        return buffer
    }
}

Does anyone knows how to convert a CMSampleBuffer to AVAudioPCMBuffer and back again? I assume there is no other way if I want to interact with AVAudioEngine, am I right?

Boost

Answer 1

Zachary Girouard OP

Oct ’19

Look into the function CMSampleBufferCopyPCMDataIntoAudioBufferList

I have my own code that uses this to stuff a CMSampleBuffer into an AudioBufferList, and you can see it here:

https://github.com/zakk4223/CocoaSplit/blob/master/CocoaSplit/CAMultiAudio/CAMultiAudioPCMPlayer.m#L203

I don't think you need to allocate any of the AudioBufferList as I believe AVAudioPCMBuffer handles that if you properly initialize it. So you can likely strip away most of the code in that function. You can initialize an AVAudioFormat with the ASBD from CMAudioFormatDescriptionGetStreamBasicDescription

So quick summary: get the ASBD and sample count from the CMSampleBuffer. Use them to create both an AVAudioFormat and AVAudioPCMBuffer. Then tell the CMSampleBuffer to copy the PCM data into the AVAudioPCMBuffer's audioBufferList

1

Answer 2

Lightning_Lion OP

Dec ’21

https://developer.apple.com/forums/profile/Zachary+Girouard

    func scheduleBuffer(_ sampleBuffer: CMSampleBuffer?) -> AVAudioPCMBuffer? {

        var sDescr: CMFormatDescription? = nil

        if let sampleBuffer = sampleBuffer {

            sDescr = CMSampleBufferGetFormatDescription(sampleBuffer)

        }

        var numSamples: CMItemCount? = nil

        if let sampleBuffer = sampleBuffer {

            numSamples = CMSampleBufferGetNumSamples(sampleBuffer)

        }



        var avFmt: AVAudioFormat? = nil

  

            avFmt = AVAudioFormat(cmAudioFormatDescription: sDescr!)

        





        var pcmBuffer: AVAudioPCMBuffer? = nil

        if let avFmt = avFmt {

            pcmBuffer = AVAudioPCMBuffer(pcmFormat: avFmt, frameCapacity: AVAudioFrameCount(UInt(numSamples ?? 0)))

        }

        pcmBuffer?.frameLength = AVAudioFrameCount(UInt(numSamples ?? 0))



        if let sampleBuffer = sampleBuffer, let mutableAudioBufferList = pcmBuffer?.mutableAudioBufferList {

            CMSampleBufferCopyPCMDataIntoAudioBufferList(sampleBuffer, at: 0, frameCount: Int32(numSamples ?? 0), into: mutableAudioBufferList)

        }

        return pcmBuffer

    }

0

Answer 3

Fred-Klein OP

Oct ’22

this might help as well https://developer.apple.com/documentation/speech/sfspeechaudiobufferrecognitionrequest/1649395-appendaudiosamplebuffer

func appendAudioSampleBuffer(_ sampleBuffer: CMSampleBuffer)

0