AVAudioCompressedBuffer allocation differences between El Capitan and Sierra

There's a difference between El Capitan and Sierra in created AVAudioCompressedBuffers. The mutableAudioBufferList's sole buffer has a zero-length mDataByteSize value on Sierra unlike El Capitan. This causes a problem when then attempting to pass this output buffer to an AVAudioConverter (-50 paramErr). It took several hours and assembly diving to track this down. So now 2015 WWDC Session 507's code in the slides is wrong. This is going to trip up a lot of people I'm sure.


Why was this changed? I don't understand the logic behind it.


Everthing else I inspect on the buffer is the same between El Cap and Sierra. This is confusing. And in addition to that, you can't simply modify the mDataByteSize value through the mutableAudioBufferList because when calling mutableAudioBufferList, it resets it back to zero. Soooo... I can't even figure out how to use it anymore. If it keeps resetting the data size to zero, and the AVAudioConverter (AudioConverterFillComplexBuffer underneath) throws paramErr whenever it's zero, then how is supposed to be used? The only way I can see of making this work is to cast the abl to be mutable and set its size like so:


UnsafeMutablePointer<AudioBufferList>(outputBuffer.audioBufferList).memory.mBuffers.mDataByteSize = UInt32(outputBuffer.packetCapacity) * UInt32(outputBuffer.maximumPacketSize)



static func AAC() -> AVAudioFormat {
  var outDesc = AudioStreamBasicDescription(
  mSampleRate: 44100, mFormatID: kAudioFormatMPEG4AAC, mFormatFlags: 0,
  mBytesPerPacket: 0, mFramesPerPacket: 0, mBytesPerFrame: 0,
  mChannelsPerFrame: 2, mBitsPerChannel: 0, mReserved: 0)
  let outFormat = AVAudioFormat(streamDescription: &outDesc)
  return outFormat
}

let outputBuffer = AVAudioCompressedBuffer(format: avconv.outputFormat, packetCapacity: 8, maximumPacketSize: avconv.maximumOutputPacketSize)

print(outputBuffer.mutableAudioBufferList.memory)

// ---- El Capitan ----------------
// (AudioBufferList) $R1 = {
//   mNumberBuffers = 1
//   mBuffers = {
//     mNumberChannels = 1
//     mDataByteSize = 12288
//     mData = 0x00000001020af400
//   }
// }
//
// ---- Sierra --------------------
// (AudioBufferList) $R0 = {
//   mNumberBuffers = 1
//   mBuffers = {
//     mNumberChannels = 1
//     mDataByteSize = 0
//     mData = 0x00000001018cac00
//   }
// }



// Example usage which comes straight from WWDC slides and no longer works on Sierra
{
  let avconv = AVAudioConverter(fromFormat: AVAudioFormat.processing(), toFormat: AVAudioFormat.AAC())
  let outputBuffer = AVAudioCompressedBuffer(format: avconv.outputFormat, packetCapacity: 8, maximumPacketSize: avconv.maximumOutputPacketSize)

  // On El Cap this was not required, but it seems the only way to make this work now.
// We must cast the non-mutable ABL because changes to the mutable one are overwritten each time mutableAudioBufferList is called.
  // UnsafeMutablePointer<AudioBufferList>(outputBuffer.audioBufferList).memory.mBuffers.mDataByteSize = UInt32(outputBuffer.packetCapacity) * UInt32(outputBuffer.maximumPacketSize)


  var nserror: NSError? = nil
  let status = avconv.convertToBuffer(outputBuffer, error: &nserror) {
    (requestedFrameCount: AVAudioPacketCount, inputStatus: UnsafeMutablePointer<AVAudioConverterInputStatus>) -> AVAudioBuffer? in

    print("callback called")
    inputStatus.memory = .NoDataNow
    return nil
  }

  if status == .Error {
    print(nserror!) /
  } else {
    print("No error")
  }
}

Accepted Reply

A fix for the -50 should be in for 16C48b which is macOS 10.12.2 beta 3 release today. The problem appears to have been in the AVAudioConverter object itself not setting byteLength to capacity before attempting a conversion.

Replies

Can you please write up a bug for this and include your test case. Just post the bug number in a reply so I can track it. I found an implementation difference in mutableAudioBufferList and discussed the change with the engineer who would likely take a closer look -- so, I'd like to ensure the issue gets routed correctly and addressed.


Edit: I filed a bug for this issue earlier today.
Not only for AVAudioConverter but to generally have
a look at how the AVAudioBuffer objects are initially created.
I will update the thread when there is a seed to test a fix with.

I think I'm getting bitten by the same bug here: When AVAudioPCMBuffer is initialized with a 2 channel interleaved format, the mutableAudioBufferList is pointing to a 1 channel noninterleaved AudioBuffer. It looks like this is different from El Cap.

A fix for the -50 should be in for 16C48b which is macOS 10.12.2 beta 3 release today. The problem appears to have been in the AVAudioConverter object itself not setting byteLength to capacity before attempting a conversion.

Thank you for the workaround of casting the audioBufferList property to an UnsafeMutablePointer! I spend quite some time trying to accomplish that with mutableAudioBufferList 😟


I filed a bug for the issue of AVAudioBuffer.mutableAudioBufferList not actually being mutable: 29840941


Edit: I just came across this message which has some useful info on working with audio buffers. Apparently (if I understood this correctly) there is a subtle difference where the audioBufferList's and mutableAudioBufferList's contained audioBuffers mDataByteSize members represent different things, either frameLength or frameCapacity.

If that is the case, the behaviour would make sense because you cannot just change the frameCapacity of the buffer. When looking at the value of a freshly created AVAudioPCMBuffer.mutableAudioBufferList.pointee.mBuffers.mDataByteSize however, it is 0 and does not seem to represent the frameCapacity.


theanalogkid, would you mind giving some guidance here? 🙂

mutableAudioBufferList returns a copy of the ABL, so you would need to modify that returned ABL directly, it's unclear if that's what you're doing. Modifying the actual audioBufferList means you're touching the real ABL. That's why the other message says "If this size is altered, you should modify the AVAudioPCMBuffer's frameLength to match."


When you set frameLengh, that's when the AVAudioPCMBuffer's underlying real ABL number of AudioBuffers mDataByteSize'es are set. You can still modify the buffer content using audioBufferList.


The reason you're seeing 0 is likely because you're never doing this.


myPCMBuffer.frameLength = myPCMBuffer.frameCapacity


The issue in this thread was really a result of a bug in the AVAudioConverter itself which has been fixed.

I'm seeing this on iOS 10.2 using Swift and this has me blocked with my work. I'm going to try the trick of casting the immutable audioBufferLost to a mutable version. I've also file directly a radar to Apple about this: 30386673. So surprising that Apple didn't thoroughly test this before releasing.

Here's my code in ObjectiveC: AudioStreamBasicDescription blah2 = audioFile.audioFormat; AVAudioFormat *format = [[AVAudioFormat alloc] initWithStreamDescription:&blah2]; AVAudioCompressedBuffer *blah = [[AVAudioCompressedBuffer alloc] initWithFormat:format packetCapacity:1 maximumPacketSize:179]; blah.packetCount = 1; blah.mutableAudioBufferList->mNumberBuffers = 1; blah.mutableAudioBufferList->mBuffers[0].mData = buffer; blah.mutableAudioBufferList->mBuffers[0].mDataByteSize = bytesRead; blah.mutableAudioBufferList->mBuffers[0].mNumberChannels = audioFile.audioFormat.mChannelsPerFrame;

What you're seeing is from the description method of the AVAudioBuffer class. It is printing out the byteLength and byteCapacity of the object or 0/179 in your example. byteLength is only ever set for PCM.


byteCapacity on the other hand is being set when you initialize the AVAudioCompressedBuffer using the values passed in as (numberOfPackets * maximumPacketSize) or 1*179 (the format has mBytesPerPacket set to 0). The packetCount should be set to reflect the number of compressed packets in the buffer. You can change this but only if it's less than or equal to the numberOfPackets you initialized the buffer with which is the capacity.


The data property returns a pointer to mBuffers[0].mData allocated to contain up to numberOfPackets * maximumPacketSize data. If the data pointer was bogus then that would be a problem but I don't think that's the case going by what I see in the implementation (also by doing a quick check in a Swift playground, storing and loading some values from the data pointer as a UnsafeMutableRawPointer). No need to mess with the ABL.


I moved the report over to the engineer who maintains AVAudioBuffer, maybe a nicer way to describe this object when you print it out.

Thanks a lot; that was confusing me in trying to debug my code. I was able to verify that the data field is properly set via the following code:


    private func copyBuffer(_ buffer: [UInt8], to compressedBuffer: AVAudioCompressedBuffer, numberOfBytes: UInt32, numberOfPackets: UInt32, packetDescriptions: [AudioStreamPacketDescription]) {
       
        compressedBuffer.packetCount = numberOfPackets
        compressedBuffer.data.initializeMemory(as: UInt8.self, from: buffer, count: Int(numberOfBytes))
        var bufferPacketDescriptions = compressedBuffer.packetDescriptions
        for index in 0..<packetDescriptions.count {
            bufferPacketDescriptions?.pointee = packetDescriptions[index]
            bufferPacketDescriptions = bufferPacketDescriptions?.successor()
        }
    }


I'm still getting a -50 error from my AVAudioConverter on iOS 10.2, and I'm not sure what else could be wrong. I'm going from AAC to PCM in my conversion.


I heard from DTS that this should have been fixed on iOS 10.2....is that not true?

The -50 is unfortunately a generic error so yes, there were some -50 issues fixed (and discussed in other threads) but this one may be caused by a different issue we haven't seen yet - hard to say. Can you add a test case to the bug you filed previously that will reproduce the problem with the steps to reproduce. We'll be looking at the compressed buffer issue so having a test will help track down anything if indeed there is a different framework bug here that has to do with the AVAudioConverter. I'm copied on the bug so I can track it.


Edit: Quick clarification regarding the -50 bug that was fixed for 10.2. This was a bug in the AVAudioConverter -convertToBuffer: error: withInputFromBlock: method where the output buffers mutableBufferList (which is a copy) was being fetched before setting the size of the output buffer. This is basically the "you did not set the frameLength to reflect the buffer capacity issue). So, it is the mDataByteSize is 0 problem and the AVAudioConverter would return -50 to indicate this. The bug was fixed for the 14C63 build of iOS. iOS 10.2 is 14C92.


Edit #2: Ok, welcome to "Where is your -50 coming from?". We tracked this specific issue down and DTS opened another bug report which I related to one filed by ronak2121. There are two issues being described the exact same way but happening under different conditions. The one I mentioned above was fixed, but passing back an AVAudioCompressedBuffer to AVAudioConverter in the inputBlock needs to be looked at - the work around is ensuring you're setting mDataByteSize manually via the audioBufferList property.

Hi, I hope this is the rigth place to reply, but I think I'm having a related issue.

I'm trying to put an ACC audio pakcet into a CompressedBuffer object, but it seems I can't set mDataByteSize.

Can you elaoborate on the workaround: "the work around is ensuring you're setting mDataByteSize manually via the audioBufferList property"?


This does not work:

incomingEencodedBuffer.mutableAudioBufferList.pointee.mBuffers.mDataByteSize = UInt32(inPacket.count)

It does not change the value.

For Objective-C this is the workaround:


((AudioBufferList *)compressedBuffer.audioBufferList)->mBuffers[0].mDataByteSize = (int32_t)data.length;


As in the original post suggest, you need to cast the .audioBufferList, not the .mutableAudioBufferList.


Greetings!