`videoChat` AVAudioSession.Mode Issues on iPhone 14 Pro Max

I work on a video conferencing application, which makes use of AVAudioEngine and the videoChat AVAudioSession.Mode

This past Friday, an internal user reported an "audio cutting in and out" issue with their new iPhone 14 Pro, and I was able to reproduce the issue later that day on my iPhone 14 Pro Max. No other iOS devices running iOS 16 are exhibiting this issue.

I have narrowed down the root cause to the videoChat AVAudioSession.Mode after changing line 53 of the ViewController.swift file in Apple's "Using Voice Processing" sample project (https://developer.apple.com/documentation/avfaudio/audio_engine/audio_units/using_voice_processing) from:

try session.setCategory(.playAndRecord, options: .defaultToSpeaker)

to

try session.setCategory(.playAndRecord, mode: .videoChat, options: .defaultToSpeaker)

This only causes issues on my iPhone 14 Pro Max device, not on my iPhone 13 Pro Max, so it seems specific to the new iPhones only.

I am also seeing the following logged to the console using either device, which appears to be specific to iOS 16, but am not sure if it is related to the videoChat issue or not:

2022-09-19 08:23:20.087578-0700 AVEchoTouch[2388:1474002] [as] ATAudioSessionPropertyManager.mm:71  Invalid input size for property 1684431725
2022-09-19 08:23:20.087605-0700 AVEchoTouch[2388:1474002] [as] ATAudioSessionPropertyManager.mm:225  Invalid input size for property 1684431725

I am assuming 1684431725 is 'dfcm' but I am not sure what Audio Session Property that might be.

The two errors are possibly related to using kAudioUnitSubType_VoiceProcessingIO type and they are not logged with kAudioUnitSubType_RemoteIO.

we have the same issue.did you solve it?

Same issue here, the problem only exists on iPhone 14. We sent a very small app to Apple showing the issue, and the only response we got was 'works as designed'. Clearly not the case.

We set our callback size to 1024 samples, and with RemoteIO we get exactly 1024 each time. However, with VoiceProcessingIO we get mostly 960 with the occasional 1440 callback, and it is at the 960/1440 callback boundary where the render errors occur, resulting in loss of microphone data.

Yes, we meet the same issue too, which only happens on iPhone 14 series, @stefanator do you have fixed this problem now ?

I also work on an app with kind of "video conferencing" use case. We use WebRTC for this use case and, on iPhone 14 (at least Pro, didn't have the chance to test with other devices) we have sound issues as soon as the WebRTC connection to another device is up and running.

Did someone found how to solve this ?

Facing similar issues on an iPhone 14 Pro. Did you manage to find the root cause?

We are seeing the same issue on iPhone 14 and basically iOS app is broken at this point.

Facing the same issue on the 14. When I look at AudioStreamBasicDescription of the CMSampleBuffer that are received from videoChat mode on 14, I get this

Optional(<CMAudioFormatDescription 0x283384820 [0x1f2fa8580]> {
  mediaType:'soun' 
  mediaSubType:'lpcm' 
  mediaSpecific: {
    ASBD: {
      mSampleRate: 44100.000000 
      mFormatID: 'lpcm' 
      mFormatFlags: 0xc 
      mBytesPerPacket: 8 
      mFramesPerPacket: 1 
      mBytesPerFrame: 8 
      mChannelsPerFrame: 4 
      mBitsPerChannel: 16   } 
    cookie: {(null)} 
    ACL: {Mono}
    FormatList Array: {
      Index: 0 
      ChannelLayoutTag: 0x640001 
      ASBD: {
      mSampleRate: 44100.000000 
      mFormatID: 'lpcm' 
      mFormatFlags: 0xc 
      mBytesPerPacket: 8 
      mFramesPerPacket: 1 
      mBytesPerFrame: 8 
      mChannelsPerFrame: 4 
      mBitsPerChannel: 16   }} 
  } 
  extensions: {(null)}
})

When I initialize audio session in default mode, the CMAudioFormatDescription of audio samples is

Optional(<CMAudioFormatDescription 0x282559540 [0x2021f4580]> {
	mediaType:'soun' 
	mediaSubType:'lpcm' 
	mediaSpecific: {
		ASBD: {
			mSampleRate: 44100.000000 
			mFormatID: 'lpcm' 
			mFormatFlags: 0xc 
			mBytesPerPacket: 2 
			mFramesPerPacket: 1 
			mBytesPerFrame: 2 
			mChannelsPerFrame: 1 
			mBitsPerChannel: 16 	} 
		cookie: {(null)} 
		ACL: {Mono}
		FormatList Array: {
			Index: 0 
			ChannelLayoutTag: 0x640001 
			ASBD: {
			mSampleRate: 44100.000000 
			mFormatID: 'lpcm' 
			mFormatFlags: 0xc 
			mBytesPerPacket: 2 
			mFramesPerPacket: 1 
			mBytesPerFrame: 2 
			mChannelsPerFrame: 1 
			mBitsPerChannel: 16 	}} 
	} 
	extensions: {(null)}
})

This shows that audio session is sending CMSampleBuffer objects that are now fundamentally different (this new format is not handled by AVAssetWriter and I get an error upon calling AVAssetWriter.append.

This shows that audio session is sending CMSampleBuffer objects that are now fundamentally different (this new format is not handled by AVAssetWriter and I get an error upon calling AVAssetWriter.append

Hi @maverick1979, did you check this info with other devices than iPhone 14 series ?

Because I already checked this data and it seems on my side that this behavior is the same on iPhone 11 / 12 / 13 series but there is no audio issue on these series, while there is one on 14 series.

We are experiencing an audio issue on iPhone 14 when using Agora SDK with custom audio recording using AVCaptureSession and AVAssetWriter. There is no problem on iPhone 13 and older devices. Also interesting to note is that when an iPhone 14 device is the only device in the call, no error happens. The error happens when there is another device in the call sending audio to iPhone 14. We think that the problem happens when iPhone 14 speakerphone is open and at the same time as microphone from another AVCaptureSession is trying to capture samples. We think the problem we are observing is related to the issue mentioned in this thread.

Here is a link to a sample app and detailed steps to reproduce the issue: https://www.dropbox.com/sh/5486wrisnv6hdqu/AACNqKwbTNJhrvwE0EjTWzfza?dl=0

@apple folks this is blocking our app launch and risking a good deal of investment. Would appreciate any help here.

@stefanator I just downloaded the app and tried on iOS 16.4. It seems that Apple has fixed this issue since I did not observe audio cutting in and out. Can you confirm?

`videoChat` AVAudioSession.Mode Issues on iPhone 14 Pro Max
 
 
Q