Post not yet marked as solved
I need to know if there is a hard limit to add the number of tracks to AVMutableComposition using the API addMutableTrackWithMediaType? That is, if I want to make a collage of 9 videos, I understand I will need to add 9 tracks to AVMutableComposition and use appropriate AVCompositionInstructions to compose the video. How do I know the maximum number of tracks that can be added simultaneously and it is supported by the system? Or is this number device dependent? I read the number of simultaneous decoders that can be run is 16 in iOS, but still not in official documents.
Post not yet marked as solved
This is a strange bug I am encountering on iOS 14 and it is very hard to replicate in a stand alone project. I sometimes get the following glitch when my app loads on iOS 14 betas that never happened on previous iOS versions. The attached gif shows the issue.
GIF - https://i.stack.imgur.com/qvaE8.gif
What I have is a subview called topPanel.
@IBOutlet weak var topPanel: UIView!
It is set to white color with alpha 1 in the Storyboard.
Then in viewDidLoad, I set it to clear color as follows:
topPanel.backgroundColor = UIColor.white.withAlphaComponent(0.0)
Next, in other places in code, such as viewDidLayoutSubviews:
> override func viewDidLayoutSubviews() {
		super.viewDidLayoutSubviews()
		
		self.cameraUI = CameraUIType.singleCam
}
}
	 private var topPanelState = TopPanelState.none {
		didSet {
				
				
private var cameraUI = CameraUIType.singleCam {
		didSet {
				
				DispatchQueue.main.async {
						self.topPanelState = .none
				}
				
				
				switch topPanelState {
						
				case .none:
						topPanel.isHidden = false //Setting this to true doesn't cause bug
				 ...
			 }
			}
Any ideas where this glitch could be coming from?
```
Post not yet marked as solved
Is there a way to remove background noise from audio samples coming from microphone(RemoteIO Unit) like Apple does it in AirPods during phone call? Which AudioUnit is most effective to achieve this? Does the Phone app uses voice processing Audio unit rather than RemoteIO?
Post not yet marked as solved
Dear AVFoundation Engineers, I get this error which is reproducible sometimes while using AVAssetWriter to record in multicamera session.
Error -11800
Localized Description: "The operation could not be completed"
Localized Failure Reason: "An unknown error occurred -16364"
And I checked the timestamps for consistency. Here are the timestamps logged in order before they are appended to AVAssetWriter. I am appending trailing log of last few frames timestamps.
2020-08-12 14:41:29.255527+0530 MyApp[1474:1750168] Frame time 126866.62201475
2020-08-12 14:41:29.282100+0530 MyApp[1474:1750174] Frame time 126866.655337833
2020-08-12 14:41:29.313364+0530 MyApp[1474:1750168] Frame time 126866.688661083
2020-08-12 14:41:29.355942+0530 MyApp[1474:1750231] Frame time 126866.721984166
2020-08-12 14:41:29.387164+0530 MyApp[1474:1750174] Frame time 126866.755307333
2020-08-12 14:41:29.423126+0530 MyApp[1474:1750174] Frame time 126866.788630375
2020-08-12 14:41:29.452698+0530 MyApp[1474:1750172] Frame time 126866.821953791
2020-08-12 14:41:29.490752+0530 MyApp[1474:1750175] Frame time 126866.855276791
2020-08-12 14:41:29.518082+0530 MyApp[1474:1750168] Frame time 126866.888599958
2020-08-12 14:41:29.553148+0530 MyApp[1474:1750175] Frame time 126866.921922916
2020-08-12 14:41:29.584810+0530 MyApp[1474:1750168] Frame time 126866.955246333
2020-08-12 14:41:29.620997+0530 MyApp[1474:1750174] Frame time 126866.988569333
2020-08-12 14:41:29.650584+0530 MyApp[1474:1750168] Frame time 126867.021892291
2020-08-12 14:41:29.684804+0530 MyApp[1474:1750175] Frame time 126867.05521575
2020-08-12 14:41:29.724279+0530 MyApp[1474:1750172] Frame time 126867.08853875
2020-08-12 14:41:29.747264+0530 MyApp[1474:1750175] Frame time 126867.121862208
2020-08-12 14:41:29.781779+0530 MyApp[1474:1750172] Frame time 126867.155184916
2020-08-12 14:41:29.817972+0530 MyApp[1474:1750175] Frame time 126867.18850825
2020-08-12 14:41:29.851997+0530 MyApp[1474:1750172] Frame time 126867.221831458
2020-08-12 14:41:29.885758+0530 MyApp[1474:1750175] Frame time 126867.2551545
2020-08-12 14:41:29.916920+0530 MyApp[1474:1750172] Frame time 126867.288477791
2020-08-12 14:41:29.963321+0530 MyApp[1474:1750172] Frame time 126867.31711775
2020-08-12 14:41:29.992338+0530 MyApp[1474:1750172] Frame time 126867.350480875
2020-08-12 14:41:30.029164+0530 MyApp[1474:1750168] Frame time 126867.383843583
2020-08-12 14:41:30.068155+0530 MyApp[1474:1750175] Frame time 126867.417207541
2020-08-12 14:41:30.095844+0530 MyApp[1474:1750175] Frame time 126867.450570833
2020-08-12 14:41:30.133841+0530 MyApp[1474:1750168] Frame time 126867.483933791
2020-08-12 14:41:30.168765+0530 MyApp[1474:1750175] Frame time 126867.517297208
2020-08-12 14:41:30.198893+0530 MyApp[1474:1750168] Frame time 126867.550660416
2020-08-12 14:41:30.235167+0530 MyApp[1474:1750168] Frame time 126867.584023666
2020-08-12 14:41:30.257920+0530 MyApp[1474:1750168] Frame time 126867.617387208
2020-08-12 14:41:30.259389+0530 MyApp[1474:1750009] Error Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-16364), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x11eb6f2d0 {Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"}}, Info ["NSLocalizedDescription": The operation could not be completed, "NSLocalizedFailureReason": An unknown error occurred (-16364), "NSUnderlyingError": Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"]
Post not yet marked as solved
Dear AVFoundation Engineers, I get this error which is reproducible sometimes while using AVAssetWriter to record in multicamera session.
Error -11800
Localized Description: "The operation could not be completed"
Localized Failure Reason: "An unknown error occurred -16364"
Dear Apple Engineers,
This is a strange bug of iOS 14 beta 3. The following line added to any UIViewController hangs auto rotation *forever* in XCode debugger. public static let metalDevice:MTLDevice? = MTLCreateSystemDefaultDevice() //Freezes iOS 14 device autorotation forever
Now even if you remove this line, nothing happens. The device will never autorotate when connected to XCode debugger. It however autorotates when not connected to debugger. The only way is to remove the line, compress it in zip file, delete the project and uncompress the zip file and build again.
What is the known workaround? I shall be filing bug report soon.
Post not yet marked as solved
I am seeing an unusual autorotation bug in iOS 14 (beta 3). I have an XCode project with some AVFoundation (AVCaptureSession) & Metal code to setup camera and rendering of camera frames. Here is the strange thing about bug.
Build the project with XCode 12 and run on iOS 14 device. The view controller doesn't autorotate when connected to debugger. When the app is opened without debugger, autorotation happens,
Once Autorotation is blocked in debugger, it gets blocked forever for the XCode project. Meaning, if I change the initial view controller to some dummy view controller, it too fails to autorotate!!! Only way is to make a new XCode project to reproduce the issue. Deleting/Reinstalling the app, clean build, nothing helps.
This does not happen on iOS 13 devices.
Reproducible test case is very difficult to make as there is lot of code. I tried trimming a lot of code but then it is not reproducible. And even to find the culprit code, I will need to make a new XCode project everytime as the project once dead is dead forever!
Is anyone aware of similar issue? Any inputs would be appreciated.
Post not yet marked as solved
I get the following compilation errors on XCode 12 that are not there when building the same project with XCode 11.
preferredAutoMicrophoneLocation = AVAudioSession.Location.orientationFront
Error: Type 'AVAudioSession.Location' has no member 'orientationFront'
Has something changed in the new iOS SDK?
Post not yet marked as solved
XCode 12 is making debugging application impossible if you have Metal code such as this
renderEncoder?.setViewport(MTLViewport(originX: 0, originY: 0, width: Double(destWidth), height: Double(destHeight), znear: -1, zfar: 1))
[MTLDebugRenderCommandEncoder setViewport:]:2667: failed assertion `Viewport Validation
viewport.znear(-1) must be between 0.0f and 1.0f
This is not a problem when the app is not connected to debugger. How do I fix this?
Post not yet marked as solved
Some iPhone 7 plus (and only these device models) are throwing an error
let audioDevice = AVCaptureDevice.default(for: AVMediaType.audio)
if audioDevice == nil {
showError("Nil Audio Device")
}
I need to understand under what circumstances could this be nil? Assuming user gave permission for Microphone.
Post not yet marked as solved
I installed iOS 14 beta, then removed the beta profile from settings and now if I re-download the beta profile, it downloads iOS 13 & iPad OS13 beta. Looks like a bug, but how do I find the iOS 14 beta profile?
Post not yet marked as solved
On iOS, I don't see a way to create a CMClock that is synchronised to Audio input source currently in use by RemoteIO unit, API is only for Audio output. Is there a way to access the audio clock for RemoteIO unit?
Post not yet marked as solved
The audio samples coming from audio connection added to AVCaptureMultiCamSession are very low power compared to AVCaptureSession. This can be easily seen by printing averagePower of audio channels in sample code of AVMultiCamPip & AVCam and comparing. Why is it so?
Post not yet marked as solved
I am getting pixel buffers in BGRA32 format in real time through AVCaptureVideoDataOutput and I am trying to compute Histogram. Following is my code to process pixel buffers to get histogram:CVPixelBufferLockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);
vImage_Error err;
vImage_Buffer buffer;
buffer.data = (unsigned char *)CVPixelBufferGetBaseAddress( pixelBuffer );
buffer.rowBytes = CVPixelBufferGetBytesPerRow( pixelBuffer );
buffer.width = CVPixelBufferGetWidth( pixelBuffer );
buffer.height = CVPixelBufferGetHeight( pixelBuffer );
vImageCVImageFormatRef vformat = vImageCVImageFormat_CreateWithCVPixelBuffer( pixelBuffer );
vImageBuffer_InitWithCVPixelBuffer(&buffer, NULL, pixelBuffer, vformat, NULL, kvImageNoAllocate);
vImagePixelCount alpha[256];
vImagePixelCount red[256];
vImagePixelCount green[256];
vImagePixelCount blue[256];
vImagePixelCount *histogram[4] = {blue, green, red, alpha};
err = vImageHistogramCalculation_ARGB8888(&buffer, histogram, kvImageNoFlags);
CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);However, I get a warning "Null passed to a callee that requires a non-null argument" for the second argument passed to vImageBuffer_InitWithCVPixelBuffer. All I want is to initialize and process the vImage and compute the histogram with minimal format conversions. Is there a passthrough way to convert pixel buffer to vImage which preserves BGRA32 format?
Post not yet marked as solved
I am dynamically switching between AVCaptureMultiCamSession & AVCaptureSession and here is the problem I face. I add back camera to MultiCam session as demonstrated in AVMultiCamPIP sample code and it works. Next I deallocate this session and create an AVCaptureSessio. But when I add the back camera device to AVCaptureSession & set the sessionPreset to 4K (as done in AVCam sample code), I get this error: Capture session runtime error: related decl 'e' for AVError(_nsError: Error Domain=AVFoundationErrorDomain Code=-11873 "Cannot Record" UserInfo={NSLocalizedDescription=Cannot Record, AVErrorDeviceKey=<AVCaptureFigVideoDevice: 0x15d0074f0 [Back Camera][com.apple.avfoundation.avcapturedevice.built-in_video:0]>, NSLocalizedRecoverySuggestion=Reset the camera to a supported active format, then restart the session., NSLocalizedFailureReason=The camera's active format is unsupported by this session.})If on the other hand I set the sessionPreset to 1080p, I get a different error - "Capture session was interrupted (video device is in use by another client)".I even tried to set the activeFormat of device before adding it to session, but no difference. Wondering what could be the problem.