Hi, Apple's engineer.
Hoping that you can reply to this one.
We're developing a Text-to-Speak app. Everything went well until the IOS got upgraded to 18.
AVSpeechSynthesisVoice(language: "zh-CN") is running well under IOS 16 AND IOS 17. It speaks Mandarin correctly.
In IOS 18, we noticed that Siri's Language setting interrupted the performance of AVSpeechSynthesisVoice. It plays Cantonese instead of Mandarin.
Buggy language setting in Siri that affects the AVSpeechSynthesisVoice :
Chinese (Cantonese - China mainland)
Chinese (Cantonese -Hong Kong)
AVFoundation
RSS for tagWork with audiovisual assets, control device cameras, process audio, and configure system audio interactions using AVFoundation.
Posts under AVFoundation tag
200 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I want to apply a SCNTechnique pipeline to the camera feed. To achieve this, I want to bring the camera input into the SceneKit world.
The perfects API seems to be:
let captureDevice = …
scnScene.background.contents = captureDevice
This is demonstrated in "SceneKit: What's New" (WWDC17) (at 44m19s) and is mentioned in the documentation of SCNMaterialProperty's contents.
Instead of showing camera feed, it crashes with these messages:
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[AVCaptureVideoDataOutput setVideoSettings:] Unsupported pixel format type - use -availableVideoCVPixelFormatTypes'
*** First throw call stack:
(0x18993c7cc <REDACTED> 0x211e18488)
libc++abi: terminating due to uncaught exception of type NSException
Please advise.
STEPS TO REPRODUCE
Create a new Xcode project, starting from the SceneKit game template.
Add Info.plist entry for NSCameraUsageDescription.
Add a capture device property to GameViewController:
class GameViewController: UIViewController {
let captureDevice = AVCaptureDevice.default(for: .video)
Set the background contents:
scene.background.contents = captureDevice
Run the app on device.
PLATFORM AND VERSION
iOS
Development environment: Xcode 16.1, macOS 15.0.1. Run-time configuration: iOS 18.1
We have a universal iOS/tvOS app that also supports iOS App on Mac.
In our AVPlayer-based video player we support AirPlay with AVRouteDetector and AVRoutePickerView. We play HLS streams.
When we try to AirPlay from an iOS device to an Apple TV or a Mac that has our app installed, it doesn't work. The receiver is marked as active in the route picker UI but the video doesn't show up on the receiver and playback stops.
When our app isn't installed on the receiver device, everything works as expected.
Has anyone encountered the same issue? Any solutions available for this?
We're integrating a web based group calling application within a native iOS application and finding that every time a CallKit session gets fully established the web based media streams break, rendering as gray with no audio.
Up to iOS 18 we worked around it by not fulfilling the call start action but that's no longer an option as the audio stopped getting automatically redirected to the speakers. We would now need the CXProvider's didActivateAudioSession callback but that would break the video.
The sample project loads up a simple webpage in a WKWebView which contains a video tag streaming the media from the device's camera.
At the same time it sets up a new CallKit session by requesting and fulfilling a CXStartCallAction transaction.
You will notice that the media doesn't render and, if you are to follow the warnings we left, you will find that not fulfilling the CXStartCallAction fixes it.
Unfortunately that's not a workaround we can use as we need the CXProvider delegate to inform us about audio session changes so we can redirect the audio to the speaker (so the proximity sensor doesn't activate and locking the screen doesn't end the call)
Any insights or workarounds would be greatly appreciated.
I made a LockScreen ControlWidget with CameraCaptureIntent, but I found launch my main app from Control Widget, SceneDelegate will be called like below:
sceneWillEnterForeground
sceneDidBecomeActive
sceneWillResignActive
sceneDidBecomeActive
sceneWillResignActive be called, is it normal?
it make my app camera launch with a delay.
I am experiencing a bug when using a AVCapturePhotoBracketSettings object to capture a bracketed photo sequence on iPhone 16 Pro.
Specifically, when I pass in an array of exposure values: [-x, 0, +x], where x >= 3.
Specifically, the high exposure photo capture returns a black image.
STEPS TO REPRODUCE
Run the sample app I have provided on an iPhone 16 Pro
Notice that bracketed images captured where the eV is set to [-3,0,+3], [-4,0,+4], or [-5,0,+5] return a black image for the high exposure photo.
Notice that on other iOS devices (like iPhone 13 Pro), the high exposure photo is returned as high brightness as expected.
I have also added two folders in the sample project that show screenshots of the bug: iPhone13Pro & iPhone16Pro
Sample Project:
https://www.icloud.com/iclouddrive/090O_68Z0Nh2UOxmPRwu56Tmw#Focused16ProBracketedCaptureBug
Hey,
I am fairly new to working with AVFoundation etc. As far as I could research on my own, if I want to get metadata from let's say a .m4a audio file, I have to get the data and then create an AVAsset. My files are all on local servers and therefore I would not be able to just pass in the URL.
The extraction of the metadata works fine - however those AVAssets create a huge overhead in storage consumption. To my knowledge the data instances of each audio file and AVAsset should only live inside the function I call to extract the metadata, however those data/AVAsset instances still live on on storage as I can clearly see that the app's file size increases by multiple Gigabytes (equal to the library size I test with). However, the only data that I purposefully save with SwiftData is the album artwork.
Is this normal behavior for AVAssets or am I missing some detail?
PS. If I forgot to mention something important, please ask. This is my first ever post, so I'm not too sure what is worth mentioning.
Thank you in advance!
Denis
The media services used for HLS streaming in an AVPlayer seem to crash if your segments are too large.
Anything over 20Mbps seems to cause a crash. I have tried adjusting the segment length to 1 second also and it didn't help.
I am remuxing Dolby Vision and HDR video and want to avoid transcoding and losing any metadata. However the segments are too large.
Is there a workaround for this? Otherwise it seems AVFoundation is not suited to high bitrate HLS and I should be using MPV or similar.
Hello,
I have a problem reading a 2D data matrix type code with a camera. In the application, I use AVFoundation to operate the camera and work with 2D codes, and in the vast majority there is no problem with loading. Nothing special.
I originally thought it might be a problem in my code, but I got the same result when I tried with the Camera app integrated in IOS. It can be seen that only the LiveText API for text recognition worked.
But I am attaching the code with which the camera has a problem, even though the code looks perfectly fine at first glance. A classic handheld 2D code reader will read the code just fine.
Can someone please explain to me why the camera, which normally reads these codes at the speed of light, sometimes has a problem with the codes?
Thank you
[Personal Information Edited by Moderator]
There are different microphones that can be connected via a 3.5-inch jack or via USB or via Bluetooth, the behavior is the same.
There is a code that gets access to the microphone (connected to the 3.5-inch audio jack) and starts an audio capture session. At the same time, the microphone use icon starts to be displayed. The capture of the audio device (microphone) continues for a few seconds, then the session stops, the microphone use icon disappears, then there is a pause of a few seconds, and then a second attempt is made to access the same microphone and start an audio capture session. At the same time, the microphone use icon is displayed again. After a few seconds, access to the microphone stops and the audio capture session stops, after which the microphone access icon disappears.
Next, we will try to perform the same actions, but after the first stop of access to the microphone, we will try to pull the microphone plug out of the connector and insert it back before trying to start the second session. In this case, the second attempt to access begins, the running part of the program does not return errors, but the microphone access icon is not displayed, and this is the problem. After the program is completed and restarted, this icon is displayed again.
This problem is only the tip of the iceberg, since it manifests itself in the fact that it is not possible to record sound from the audio microphone after reconnecting the microphone until the program is restarted.
Is this normal behavior of the AVFoundation framework? Is it possible to somehow make it so that after reconnecting the microphone, access to it occurs correctly and the usage indicator is displayed? What additional actions should the programmer perform in this case? Is there a description of this behavior somewhere in the documentation?
Below is the code to demonstrate the described behavior.
I am also attaching an example of the microphone usage indicator icon.
Computer description: MacBook Pro 13-inch 2020 Intel Core i7 macOS Sequoia 15.1.
#include <chrono>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <thread>
#include <AVFoundation/AVFoundation.h>
#include <Foundation/NSString.h>
#include <Foundation/NSURL.h>
AVCaptureSession* m_captureSession = nullptr;
AVCaptureDeviceInput* m_audioInput = nullptr;
AVCaptureAudioDataOutput* m_audioOutput = nullptr;
std::condition_variable conditionVariable;
std::mutex mutex;
bool responseToAccessRequestReceived = false;
void receiveResponse()
{
std::lock_guard<std::mutex> lock(mutex);
responseToAccessRequestReceived = true;
conditionVariable.notify_one();
}
void waitForResponse()
{
std::unique_lock<std::mutex> lock(mutex);
conditionVariable.wait(lock, [] { return responseToAccessRequestReceived; });
}
void requestPermissions()
{
responseToAccessRequestReceived = false;
[AVCaptureDevice requestAccessForMediaType:AVMediaTypeAudio completionHandler:^(BOOL granted)
{
const auto status = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeAudio];
std::cout << "Request completion handler granted: " << (int)granted << ", status: " << status << std::endl;
receiveResponse();
}];
waitForResponse();
}
void timer(int timeSec)
{
for (auto timeRemaining = timeSec; timeRemaining > 0; --timeRemaining)
{
std::cout << "Timer, remaining time: " << timeRemaining << "s" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
bool updateAudioInput()
{
[m_captureSession beginConfiguration];
if (m_audioOutput)
{
AVCaptureConnection *lastConnection = [m_audioOutput connectionWithMediaType:AVMediaTypeAudio];
[m_captureSession removeConnection:lastConnection];
}
if (m_audioInput)
{
[m_captureSession removeInput:m_audioInput];
[m_audioInput release];
m_audioInput = nullptr;
}
AVCaptureDevice* audioInputDevice = [AVCaptureDevice deviceWithUniqueID: [NSString stringWithUTF8String: "BuiltInHeadphoneInputDevice"]];
if (!audioInputDevice)
{
std::cout << "Error input audio device creating" << std::endl;
return false;
}
// m_audioInput = [AVCaptureDeviceInput deviceInputWithDevice:audioInputDevice error:nil];
// NSError *error = nil;
NSError *error = [[NSError alloc] init];
m_audioInput = [AVCaptureDeviceInput deviceInputWithDevice:audioInputDevice error:&error];
if (error)
{
const auto code = [error code];
const auto domain = [error domain];
const char* domainC = domain ? [domain UTF8String] : nullptr;
std::cout << code << " " << domainC << std::endl;
}
if (m_audioInput && [m_captureSession canAddInput:m_audioInput]) {
[m_audioInput retain];
[m_captureSession addInput:m_audioInput];
}
else
{
std::cout << "Failed to create audio device input" << std::endl;
return false;
}
if (!m_audioOutput)
{
m_audioOutput = [[AVCaptureAudioDataOutput alloc] init];
if (m_audioOutput && [m_captureSession canAddOutput:m_audioOutput])
{
[m_captureSession addOutput:m_audioOutput];
}
else
{
std::cout << "Failed to add audio output" << std::endl;
return false;
}
}
[m_captureSession commitConfiguration];
return true;
}
void start()
{
std::cout << "Starting..." << std::endl;
const bool updatingResult = updateAudioInput();
if (!updatingResult)
{
std::cout << "Error, while updating audio input" << std::endl;
return;
}
[m_captureSession startRunning];
}
void stop()
{
std::cout << "Stopping..." << std::endl;
[m_captureSession stopRunning];
}
int main()
{
requestPermissions();
m_captureSession = [[AVCaptureSession alloc] init];
start();
timer(5);
stop();
timer(10);
start();
timer(5);
stop();
}
I’m using AVFoundation in my iPhone application to encode a video in MP4 format with H.264, which can then be shared or exported.
Do I need to pay a license for using the H.264 format to MPEG LA? Or are these fees already covered by Apple?
I’ve read articles suggesting that Apple covers these fees when encoding is done through its native APIs (or via its dedicated encoding hardware components), but I haven’t found any explicit confirmation of this point in the various documentation or contracts... Did I miss something?
Our capture application records system audio via HAL plugin, however, with the latest macOS 15 Sequoia, all audio buffer values are zero.
I am attaching sample code that replicates the problem. Compile as a Command Line Tool application with Xcode.
STEPS TO REPRODUCE
Install BlackHole 2ch audio driver:
https://existential.audio/blackhole/download/?code=1579271348
Start some system audio, e.g. YouTube.
Compile and run the sample application.
On macOS up to Sonoma, you will hear audio via loopback and see audio values in the debug/console window.
On macOS Sequoia, you will not hear audio and the audio values are 0.
#import <AVFoundation/AVFoundation.h>
#import <CoreAudio/CoreAudio.h>
#define BLACKHOLE_UID @"BlackHole2ch_UID"
#define DEFAULT_OUTPUT_UID @"BuiltInSpeakerDevice"
@interface AudioCaptureDelegate : NSObject <AVCaptureAudioDataOutputSampleBufferDelegate>
@end
void setDefaultAudioDevice(NSString *deviceUID);
@implementation AudioCaptureDelegate
// receive samples from CoreAudio/HAL driver and print amplitute values for testing
// this is where samples would normally be copied and passed downstream for further processing which
// is not needed in this simple sample application
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
// Access the audio data in the sample buffer
CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
if (!blockBuffer) {
NSLog(@"No audio data in the sample buffer.");
return;
}
size_t length;
char *data;
CMBlockBufferGetDataPointer(blockBuffer, 0, NULL, &length, &data);
// Process the audio samples to calculate the average amplitude
int16_t *samples = (int16_t *)data;
size_t sampleCount = length / sizeof(int16_t);
int64_t sum = 0;
for (size_t i = 0; i < sampleCount; i++) {
sum += abs(samples[i]);
}
// Calculate and log the average amplitude
float averageAmplitude = (float)sum / sampleCount;
NSLog(@"Average Amplitude: %f", averageAmplitude);
}
@end
// set the default audio device to Blackhole while testing or speakers when done
// called by main
void setDefaultAudioDevice(NSString *deviceUID) {
AudioObjectPropertyAddress address;
AudioDeviceID deviceID = kAudioObjectUnknown;
UInt32 size;
CFStringRef uidString = (__bridge CFStringRef)deviceUID;
// Gets the device corresponding to the given UID.
AudioValueTranslation translation;
translation.mInputData = &uidString;
translation.mInputDataSize = sizeof(uidString);
translation.mOutputData = &deviceID;
translation.mOutputDataSize = sizeof(deviceID);
size = sizeof(translation);
address.mSelector = kAudioHardwarePropertyDeviceForUID;
address.mScope = kAudioObjectPropertyScopeGlobal; //????
address.mElement = kAudioObjectPropertyElementMain;
OSStatus status = AudioObjectGetPropertyData(kAudioObjectSystemObject, &address, 0, NULL, &size, &translation);
if (status != noErr) {
NSLog(@"Error: Could not retrieve audio device ID for UID %@. Status code: %d", deviceUID, (int)status);
return;
}
AudioObjectPropertyAddress propertyAddress;
propertyAddress.mSelector = kAudioHardwarePropertyDefaultOutputDevice;
propertyAddress.mScope = kAudioObjectPropertyScopeGlobal;
status = AudioObjectSetPropertyData(kAudioObjectSystemObject, &propertyAddress, 0, NULL, sizeof(AudioDeviceID), &deviceID);
if (status == noErr) {
NSLog(@"Default audio device set to %@", deviceUID);
} else {
NSLog(@"Failed to set default audio device: %d", status);
}
}
// sets Blackhole device as default and configures it as AVCatureDeviceInput
// sets the speakers as loopback so we can hear what is being captured
// sets up queue to receive capture samples
// runs session for 30 seconds, then restores speakers as default output
int main(int argc, const char * argv[]) {
@autoreleasepool {
// Create the capture session
AVCaptureSession *session = [[AVCaptureSession alloc] init];
// Select the audio device
AVCaptureDevice *audioDevice = nil;
NSString *audioDriverUID = nil;
audioDriverUID = BLACKHOLE_UID;
setDefaultAudioDevice(audioDriverUID);
audioDevice = [AVCaptureDevice deviceWithUniqueID:audioDriverUID];
if (!audioDevice) {
NSLog(@"Audio device %s not found!", [audioDriverUID UTF8String]);
return -1;
} else {
NSLog(@"Using Audio device: %s", [audioDriverUID UTF8String]);
}
// Configure the audio input with the selected device (Blackhole)
NSError *error = nil;
AVCaptureDeviceInput *audioInput = [AVCaptureDeviceInput deviceInputWithDevice:audioDevice error:&error];
if (error || !audioInput) {
NSLog(@"Failed to create audio input: %@", error);
return -1;
}
[session addInput:audioInput];
// Configure the audio data output
AVCaptureAudioDataOutput *audioOutput = [[AVCaptureAudioDataOutput alloc] init];
AudioCaptureDelegate *delegate = [[AudioCaptureDelegate alloc] init];
dispatch_queue_t queue = dispatch_queue_create("AudioCaptureQueue", NULL);
[audioOutput setSampleBufferDelegate:delegate queue:queue];
[session addOutput:audioOutput];
// Set audio settings
NSDictionary *audioSettings = @{
AVFormatIDKey: @(kAudioFormatLinearPCM),
AVSampleRateKey: @48000,
AVNumberOfChannelsKey: @2,
AVLinearPCMBitDepthKey: @16,
AVLinearPCMIsFloatKey: @NO,
AVLinearPCMIsNonInterleaved: @NO
};
[audioOutput setAudioSettings:audioSettings];
AVCaptureAudioPreviewOutput * loopback_output = nil;
loopback_output = [[AVCaptureAudioPreviewOutput alloc] init];
loopback_output.volume = 1.0;
loopback_output.outputDeviceUniqueID = DEFAULT_OUTPUT_UID;
[session addOutput:loopback_output];
const char *deviceID = loopback_output.outputDeviceUniqueID ? [loopback_output.outputDeviceUniqueID UTF8String] : "nil";
NSLog(@"session addOutput for preview/loopback: %s", deviceID);
// Start the session
[session startRunning];
NSLog(@"Capturing audio data for 30 seconds...");
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:30.0]];
// Stop the session
[session stopRunning];
NSLog(@"Capture session stopped.");
setDefaultAudioDevice(DEFAULT_OUTPUT_UID);
}
return 0;
}
I've requested the authentication in my main app.
PHPhotoLibrary.requestAuthorization(for: .readWrite) { status in }
Add the privacy description in both the main app and the extension.
But No matter the device is locked or unlocked. When I call
let fetchResult = PHAsset.fetchAssets(with: .image, options: nil)
let count = fetchResult.count
the count is always zero, even after a new photo is saved to the album in the same session.
Observing 4K playback issues on tvOS 18. Encountering HTTP 416 (Range Not Satisfiable) errors when the player attempts to request byte ranges that are outside the available data on the server. This leads to fatal playback error resulting in the error
CoreMediaErrorDomain error -12939 - HTTP 416: Requested Range Not Satisfiable
Notably, there are no customizations or modifications to the standard AVPlayerViewController on tvOS player.
AVPlayer is trying to request the resource of length equals 739 bytes with an invalid byte range (739-) request. Since the request is not satisfiable server returns with 416. Note this is only limited to tvOS 18 and we are trying to understand why AVPlayer is making this invalid request in tvOS 18 resulting in playback error.
It appears that AVAudioPlayer is maintaining a strong reference to my containing class. Here is the essential code. Pay attention to the comments.
class StethRecording: NSObject, ObservableObject, Identifiable {
let player: AVAudioPlayer?
let id = UUID()
@Published var isPlaying = false
@Published var progress = 0.0
init(file: AVAudioFile) throws {
player = try AVAudioPlayer(contentsOf: file.url)
super.init()
// I used to assign the player delegate here.
// If I do that, when I delete this object, it
// doesn't go away.
player!.prepareToPlay()
}
deinit {
// If this object doesn't go away, I leave data.
// behind. Something I don't want to do.
try? deleteAssociatedAudioFile()
}
func play() {
guard let player else { return }
// So now I have to assign the delegate whenever
// I start playing.
player.delegate = self
isPlaying = true
player.play()
startUpdateTimer()
}
func stop() {
guard let player else { return }
player.stop()
playbackConcluded()
}
// MARK: - Private Methods
private func playbackConcluded() {
isPlaying = false
stopUpdateTimer()
updateProgress()
player!.reset()
// I also have to remove the delegate when I
// stop, for any reason.
player!.delegate = nil
player!.prepareToPlay()
}
}
extension StethRecording: AVAudioPlayerDelegate {
func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool) {
playbackConcluded()
}
}
This works, but is this approach really necessary? I would expect the AVAudioPlayer to use a weak reference for the delegate. Or, am I doing something else wrong here?
So, I'm trying to create my own text-to-speech setup. Problem I'm having is whenever I do a test run, the speech gets a bit choppy at the start kind of skipping over maybe a word or a few characters.
A few details:
I've essentially built a separate class for handling the speech events.
AVSpeechSynthesizer is set up as a private variable for the class so I don't expect deallocation to be the issue. Especially since it's a problem at the start.
I've got a queue set up for what it's worth so that shouldn't be a problem.
I'd appreciate any advice.
Set 3 controls to the AVCaptureSession and remove them all. The number of controls in the session is indeed 0, but the camera controls button still shows the previous 3 controls. If it is only 3->2 or 3->1, it can be modified normally, 3->0 is not OK, 0->3 is OK.
f (self.captureControl.zoom) {
if (self.zoomScaleControl) {
self.zoomScaleControl.enabled = false;
[_session removeControl:self.zoomScaleControl];
}
AVCaptureSlider *zoomSlider = [self.captureControl.zoom fetchCaptureSlider];
[zoomSlider setActionQueue:dispatch_get_main_queue() action:^(float zoomFactor) {
@strongify(self);
if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:tryChangeZoomScale:)]) {
[self.dataOutputDelegate videoCaptureSession:self tryChangeZoomScale:zoomFactor];
}
}];
self.zoomScaleControl = zoomSlider;
} else {
if (self.zoomScaleControl) {
self.zoomScaleControl.enabled = false;
[_session removeControl:self.zoomScaleControl];
}
self.zoomScaleControl = nil;
}
if (self.captureControl.exposure) {
if (self.exposureBiasControl) {
self.exposureBiasControl.enabled = false;
[_session removeControl:self.exposureBiasControl];
}
AVCaptureSlider *exposureSlider = [self.captureControl.exposure fetchCaptureSlider];
[exposureSlider setActionQueue:dispatch_get_main_queue() action:^(float bias) {
@strongify(self);
if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:tryChangeExposureBias:)]) {
[self.dataOutputDelegate videoCaptureSession:self tryChangeExposureBias:bias];
}
}];
self.exposureBiasControl = exposureSlider;
} else {
if (self.exposureBiasControl) {
self.exposureBiasControl.enabled = false;
[_session removeControl:self.exposureBiasControl];
}
self.exposureBiasControl = nil;
}
if (self.captureControl.len) {
if (self.lenControl) {
self.lenControl.enabled = false;
[_session removeControl:self.lenControl];
}
ORLenCaptureControlCustomModel *len = self.captureControl.len;
AVCaptureIndexPicker *picker = [len fetchCaptureSlider];
[picker setActionQueue:dispatch_get_main_queue() action:^(NSInteger selectedIndex) {
@strongify(self);
if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:didChangeLenIndex:datas:)]) {
[self.dataOutputDelegate videoCaptureSession:self didChangeLenIndex:selectedIndex datas:self.captureControl.len.indexDatas];
}
}];
self.lenControl = picker;
} else {
if (self.lenControl) {
self.lenControl.enabled = false;
[_session removeControl:self.lenControl];
}
self.lenControl = nil;
}
if ([_session canAddControl:self.zoomScaleControl]) {
[_session addControl:self.zoomScaleControl];
} else {
self.zoomScaleControl = nil;
}
if ([_session canAddControl:self.lenControl]) {
[_session addControl:self.lenControl];
} else {
self.lenControl = nil;
}
if ([_session canAddControl:self.exposureBiasControl]) {
[_session addControl:self.exposureBiasControl];
} else {
self.exposureBiasControl = nil;
}
if (_session.controlsDelegate == nil) {
[_session setControlsDelegate:self queue:GetCaptureControlQueue()];
}
Here is some code I have to create an AVAudioFile instance based on Int16 samples.
let format = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: 44100.0, channels: 2, interleaved: false)!
let audioFile = try AVAudioFile(forWriting: outputURL, settings: format.settings)
When writing to the file I get the following runtime error, presumably from CoreAudio.
CABufferList.h:184 ASSERTION FAILURE [(nBytes <= buf->mDataByteSize) != 0 is false]:
I read this as a size mismatch between what is specified in the format used to create the file and the file's own internal processingFormat property, which is read-only. Here is my debugger console output showing the input format I created, along with the resulting AVAudioFile fileFormat and processingFormat properties.
(lldb) po format
<AVAudioFormat 0x300e553b0: 2 ch, 44100 Hz, Int16, deinterleaved>
(lldb) po format.settings
▿ 7 elements
▿ 0 : 2 elements
- key : "AVNumberOfChannelsKey"
- value : 2
▿ 1 : 2 elements
- key : "AVLinearPCMBitDepthKey"
- value : 16
▿ 2 : 2 elements
- key : "AVFormatIDKey"
- value : 1819304813
▿ 3 : 2 elements
- key : "AVLinearPCMIsNonInterleaved"
- value : 1
▿ 4 : 2 elements
- key : "AVLinearPCMIsBigEndianKey"
- value : 0
▿ 5 : 2 elements
- key : "AVLinearPCMIsFloatKey"
- value : 0
▿ 6 : 2 elements
- key : "AVSampleRateKey"
- value : 44100
(lldb) po audioFile.fileFormat
<AVAudioFormat 0x300ea5400: 2 ch, 44100 Hz, Int16, interleaved>
(lldb) po audioFile.processingFormat
<AVAudioFormat 0x300ea5450: 2 ch, 44100 Hz, Float32, deinterleaved>
Please note that the input format I'm using does not match either the audio file fileFormat or processingFormat properties. The file format is interleaved even though I specified de-interleaved. This makes sense to me as working with audio files that are growing is much easier and more efficient with interleaved data. The head-scratcher is the processingFormat. I specified Int16 samples and it is expecting Float32? According to the format settings dictionary, we are specifying the correct key/value pairs.
Is this expected behavior? Does Apple always insist on Float32 internally or is this a bug?
The new iPhone 16 supports spatial audio recordings in the camera app when recording videos. Is it possible to also record spatial audio without video, and is it possible for 3rd party developers to do so? If so, how do I need to configure AVAudioSession and/or AVAudioEngine to record spatial audio in my audio recording app on iPhone 16?
Hi community,
I'm trying to setup an AVAudioFormat with AVAudioPCMFormatInt16. But, i've an error :
AVAEInternal.h:125 [AUInterface.mm:539:SetFormat: ([[busArray objectAtIndexedSubscript:(NSUInteger)element] setFormat:format error:&nsErr])] returned false, error Error Domain=NSOSStatusErrorDomain Code=-10868 "(null)"
If i understand the error code 10868, the format is not correct. But, how i can use PCM Int16 format ? Here is my method :
- (void)setupAudioDecoder:(double)sampleRate audioChannels:(double)audioChannels {
if (self.isRunning) {
return;
}
self.audioEngine = [[AVAudioEngine alloc] init];
self.audioPlayerNode = [[AVAudioPlayerNode alloc] init];
[self.audioEngine attachNode:self.audioPlayerNode];
AVAudioChannelCount channelCount = (AVAudioChannelCount)audioChannels;
self.audioFormat = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatInt16
sampleRate:sampleRate
channels:channelCount
interleaved:YES];
NSLog(@"Audio Format: %@", self.audioFormat);
NSLog(@"Audio Player Node: %@", self.audioPlayerNode);
NSLog(@"Audio Engine: %@", self.audioEngine);
// Error on this line
[self.audioEngine connect:self.audioPlayerNode to:self.audioEngine.mainMixerNode format:self.audioFormat];
/**NSError *error = nil;
if (![self.audioEngine startAndReturnError:&error]) {
NSLog(@"Erreur lors de l'initialisation du moteur audio: %@", error);
return;
}
[self.audioPlayerNode play];
self.isRunning = YES;*/
}
Also, i see the audioEngine seem not running ?
Audio Engine:
________ GraphDescription ________
AVAudioEngineGraph 0x600003d55fe0: initialized = 0, running = 0, number of nodes = 1
Anyone have already use this format with AVAudioFormat ?
Thank you !