User Profile

I need to know if there is a hard limit to add the number of tracks to AVMutableComposition using the API addMutableTrackWithMediaType? That is, if I want to make a collage of 9 videos, I understand I will need to add 9 tracks to AVMutableComposition and use appropriate AVCompositionInstructions to compose the video. How do I know the maximum number of tracks that can be added simultaneously and it is supported by the system? Or is this number device dependent? I read the number of simultaneous decoders that can be run is 16 in iOS, but still not in official documents.

This is a strange bug I am encountering on iOS 14 and it is very hard to replicate in a stand alone project. I sometimes get the following glitch when my app loads on iOS 14 betas that never happened on previous iOS versions. The attached gif shows the issue. GIF - https://i.stack.imgur.com/qvaE8.gif What I have is a subview called topPanel. @IBOutlet weak var topPanel: UIView! It is set to white color with alpha 1 in the Storyboard.  Then in viewDidLoad, I set it to clear color as follows: topPanel.backgroundColor = UIColor.white.withAlphaComponent(0.0) Next, in other places in code, such as viewDidLayoutSubviews: > override func viewDidLayoutSubviews() { 		super.viewDidLayoutSubviews() 		 		self.cameraUI = CameraUIType.singleCam } } 	 private var topPanelState = TopPanelState.none { 		didSet { 				 				 private var cameraUI = CameraUIType.singleCam { 		didSet { 				 				DispatchQueue.main.async { 						self.topPanelState = .none 				} 				 				 				switch topPanelState { 						 				case .none: 						topPanel.isHidden = false //Setting this to true doesn't cause bug 				 ... 			 } 			} Any ideas where this glitch could be coming from? ```

Is there a way to remove background noise from audio samples coming from microphone(RemoteIO Unit) like Apple does it in AirPods during phone call? Which AudioUnit is most effective to achieve this? Does the Phone app uses voice processing Audio unit rather than RemoteIO?

Dear AVFoundation Engineers, I get this error which is reproducible sometimes while using AVAssetWriter to record in multicamera session. Error -11800 Localized Description: "The operation could not be completed" Localized Failure Reason: "An unknown error occurred -16364" And I checked the timestamps for consistency. Here are the timestamps logged in order before they are appended to AVAssetWriter. I am appending trailing log of last few frames timestamps. 2020-08-12 14:41:29.255527+0530 MyApp[1474:1750168] Frame time 126866.62201475 2020-08-12 14:41:29.282100+0530 MyApp[1474:1750174] Frame time 126866.655337833 2020-08-12 14:41:29.313364+0530 MyApp[1474:1750168] Frame time 126866.688661083 2020-08-12 14:41:29.355942+0530 MyApp[1474:1750231] Frame time 126866.721984166 2020-08-12 14:41:29.387164+0530 MyApp[1474:1750174] Frame time 126866.755307333 2020-08-12 14:41:29.423126+0530 MyApp[1474:1750174] Frame time 126866.788630375 2020-08-12 14:41:29.452698+0530 MyApp[1474:1750172] Frame time 126866.821953791 2020-08-12 14:41:29.490752+0530 MyApp[1474:1750175] Frame time 126866.855276791 2020-08-12 14:41:29.518082+0530 MyApp[1474:1750168] Frame time 126866.888599958 2020-08-12 14:41:29.553148+0530 MyApp[1474:1750175] Frame time 126866.921922916 2020-08-12 14:41:29.584810+0530 MyApp[1474:1750168] Frame time 126866.955246333 2020-08-12 14:41:29.620997+0530 MyApp[1474:1750174] Frame time 126866.988569333 2020-08-12 14:41:29.650584+0530 MyApp[1474:1750168] Frame time 126867.021892291 2020-08-12 14:41:29.684804+0530 MyApp[1474:1750175] Frame time 126867.05521575 2020-08-12 14:41:29.724279+0530 MyApp[1474:1750172] Frame time 126867.08853875 2020-08-12 14:41:29.747264+0530 MyApp[1474:1750175] Frame time 126867.121862208 2020-08-12 14:41:29.781779+0530 MyApp[1474:1750172] Frame time 126867.155184916 2020-08-12 14:41:29.817972+0530 MyApp[1474:1750175] Frame time 126867.18850825 2020-08-12 14:41:29.851997+0530 MyApp[1474:1750172] Frame time 126867.221831458 2020-08-12 14:41:29.885758+0530 MyApp[1474:1750175] Frame time 126867.2551545 2020-08-12 14:41:29.916920+0530 MyApp[1474:1750172] Frame time 126867.288477791 2020-08-12 14:41:29.963321+0530 MyApp[1474:1750172] Frame time 126867.31711775 2020-08-12 14:41:29.992338+0530 MyApp[1474:1750172] Frame time 126867.350480875 2020-08-12 14:41:30.029164+0530 MyApp[1474:1750168] Frame time 126867.383843583 2020-08-12 14:41:30.068155+0530 MyApp[1474:1750175] Frame time 126867.417207541 2020-08-12 14:41:30.095844+0530 MyApp[1474:1750175] Frame time 126867.450570833 2020-08-12 14:41:30.133841+0530 MyApp[1474:1750168] Frame time 126867.483933791 2020-08-12 14:41:30.168765+0530 MyApp[1474:1750175] Frame time 126867.517297208 2020-08-12 14:41:30.198893+0530 MyApp[1474:1750168] Frame time 126867.550660416 2020-08-12 14:41:30.235167+0530 MyApp[1474:1750168] Frame time 126867.584023666 2020-08-12 14:41:30.257920+0530 MyApp[1474:1750168] Frame time 126867.617387208 2020-08-12 14:41:30.259389+0530 MyApp[1474:1750009] Error Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-16364), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x11eb6f2d0 {Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"}}, Info ["NSLocalizedDescription": The operation could not be completed, "NSLocalizedFailureReason": An unknown error occurred (-16364), "NSUnderlyingError": Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"]

Dear AVFoundation Engineers, I get this error which is reproducible sometimes while using AVAssetWriter to record in multicamera session. Error -11800 Localized Description: "The operation could not be completed" Localized Failure Reason: "An unknown error occurred -16364"

I am seeing an unusual autorotation bug in iOS 14 (beta 3). I have an XCode project with some AVFoundation (AVCaptureSession) & Metal code to setup camera and rendering of camera frames. Here is the strange thing about bug. Build the project with XCode 12 and run on iOS 14 device. The view controller doesn't autorotate when connected to debugger. When the app is opened without debugger, autorotation happens, Once Autorotation is blocked in debugger, it gets blocked forever for the XCode project. Meaning, if I change the initial view controller to some dummy view controller, it too fails to autorotate!!! Only way is to make a new XCode project to reproduce the issue. Deleting/Reinstalling the app, clean build, nothing helps. This does not happen on iOS 13 devices. Reproducible test case is very difficult to make as there is lot of code. I tried trimming a lot of code but then it is not reproducible. And even to find the culprit code, I will need to make a new XCode project everytime as the project once dead is dead forever! Is anyone aware of similar issue? Any inputs would be appreciated.

I get the following compilation errors on XCode 12 that are not there when building the same project with XCode 11. preferredAutoMicrophoneLocation = AVAudioSession.Location.orientationFront Error: Type 'AVAudioSession.Location' has no member 'orientationFront' Has something changed in the new iOS SDK?

XCode 12 is making debugging application impossible if you have Metal code such as this renderEncoder?.setViewport(MTLViewport(originX: 0, originY: 0, width: Double(destWidth), height: Double(destHeight), znear: -1, zfar: 1)) [MTLDebugRenderCommandEncoder setViewport:]:2667: failed assertion `Viewport Validation viewport.znear(-1) must be between 0.0f and 1.0f This is not a problem when the app is not connected to debugger. How do I fix this?

Some iPhone 7 plus (and only these device models) are throwing an error let audioDevice = AVCaptureDevice.default(for: AVMediaType.audio) if audioDevice == nil { showError("Nil Audio Device") } I need to understand under what circumstances could this be nil? Assuming user gave permission for Microphone.

I installed iOS 14 beta, then removed the beta profile from settings and now if I re-download the beta profile, it downloads iOS 13 & iPad OS13 beta. Looks like a bug, but how do I find the iOS 14 beta profile?

On iOS, I don't see a way to create a CMClock that is synchronised to Audio input source currently in use by RemoteIO unit, API is only for Audio output. Is there a way to access the audio clock for RemoteIO unit?

The audio samples coming from audio connection added to AVCaptureMultiCamSession are very low power compared to AVCaptureSession. This can be easily seen by printing averagePower of audio channels in sample code of AVMultiCamPip & AVCam and comparing. Why is it so?

I am getting pixel buffers in BGRA32 format in real time through AVCaptureVideoDataOutput and I am trying to compute Histogram. Following is my code to process pixel buffers to get histogram:CVPixelBufferLockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly); vImage_Error err; vImage_Buffer buffer; buffer.data = (unsigned char *)CVPixelBufferGetBaseAddress( pixelBuffer ); buffer.rowBytes = CVPixelBufferGetBytesPerRow( pixelBuffer ); buffer.width = CVPixelBufferGetWidth( pixelBuffer ); buffer.height = CVPixelBufferGetHeight( pixelBuffer ); vImageCVImageFormatRef vformat = vImageCVImageFormat_CreateWithCVPixelBuffer( pixelBuffer ); vImageBuffer_InitWithCVPixelBuffer(&buffer, NULL, pixelBuffer, vformat, NULL, kvImageNoAllocate); vImagePixelCount alpha[256]; vImagePixelCount red[256]; vImagePixelCount green[256]; vImagePixelCount blue[256]; vImagePixelCount *histogram[4] = {blue, green, red, alpha}; err = vImageHistogramCalculation_ARGB8888(&buffer, histogram, kvImageNoFlags); CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);However, I get a warning "Null passed to a callee that requires a non-null argument" for the second argument passed to vImageBuffer_InitWithCVPixelBuffer. All I want is to initialize and process the vImage and compute the histogram with minimal format conversions. Is there a passthrough way to convert pixel buffer to vImage which preserves BGRA32 format?

I am dynamically switching between AVCaptureMultiCamSession & AVCaptureSession and here is the problem I face. I add back camera to MultiCam session as demonstrated in AVMultiCamPIP sample code and it works. Next I deallocate this session and create an AVCaptureSessio. But when I add the back camera device to AVCaptureSession & set the sessionPreset to 4K (as done in AVCam sample code), I get this error: Capture session runtime error: related decl 'e' for AVError(_nsError: Error Domain=AVFoundationErrorDomain Code=-11873 "Cannot Record" UserInfo={NSLocalizedDescription=Cannot Record, AVErrorDeviceKey=<AVCaptureFigVideoDevice: 0x15d0074f0 [Back Camera][com.apple.avfoundation.avcapturedevice.built-in_video:0]>, NSLocalizedRecoverySuggestion=Reset the camera to a supported active format, then restart the session., NSLocalizedFailureReason=The camera's active format is unsupported by this session.})If on the other hand I set the sessionPreset to 1080p, I get a different error - "Capture session was interrupted (video device is in use by another client)".I even tried to set the activeFormat of device before adding it to session, but no difference. Wondering what could be the problem.