Hello, Faced with a really perplexing issue. Primary problem is that sometimes I get depth and video data as expected, but at other times I don't. And sometimes I'll get both data outputs for a 4-5 frames and then it'll just stop. The source code I implemented is a modified version of the sample code provided by Apple, and interestingly enough I can't re-create this issue with the Apple sample app. So wondering what I could be doing wrong? Here's the code for setting up the capture input. preferredDepthResolution is 1280 in my case. I'm running this on an iPad Pro (6th gen). iOS version 17.0.3 (21A360). Encounter this issue on iPhone 13 Pro as well. iOS version is 17.0 (21A329) private func setupLiDARCaptureInput() throws { // Look up the LiDAR camera. guard let device = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .video, position: .back) else { throw ConfigurationError.lidarDeviceUnavailable } guard let format = (device.formats.last { format in format.formatDescription.dimensions.width == preferredWidthResolution && format.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange && format.videoSupportedFrameRateRanges.first(where: {$0.maxFrameRate >= 60}) != nil && !format.isVideoBinned && !format.supportedDepthDataFormats.isEmpty }) else { throw ConfigurationError.requiredFormatUnavailable } guard let depthFormat = (format.supportedDepthDataFormats.last { depthFormat in depthFormat.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_DepthFloat16 }) else { throw ConfigurationError.requiredFormatUnavailable } // Begin the device configuration. try device.lockForConfiguration() // Configure the device and depth formats. device.activeFormat = format device.activeDepthDataFormat = depthFormat let desc = format.formatDescription dimensions = CMVideoFormatDescriptionGetDimensions(desc) let duration = CMTime(value:1, timescale:CMTimeScale(60)) device.activeVideoMinFrameDuration = duration device.activeVideoMaxFrameDuration = duration // Finish the device configuration. device.unlockForConfiguration() self.device = device print("Selected video format: \(device.activeFormat)") print("Selected depth format: \(String(describing: device.activeDepthDataFormat))") // Add a device input to the capture session. let deviceInput = try AVCaptureDeviceInput(device: device) captureSession.addInput(deviceInput) guard let audioDevice = AVCaptureDevice.default(for: .audio) else { return } // Configure audio input - always configure audio even if isAudioEnabled is false audioDeviceInput = try! AVCaptureDeviceInput(device: audioDevice) captureSession.addInput(audioDeviceInput) deviceSystemPressureStateObservation = device.observe( \.systemPressureState, options: .new ) { _, change in guard let systemPressureState = change.newValue else { return } print("system pressure \(systemPressureState.levelAsString()) due to \(systemPressureState.factors)") } } Here's how I'm setting up the output: private func setupLiDARCaptureOutputs() { // Create an object to output video sample buffers. videoDataOutput = AVCaptureVideoDataOutput() captureSession.addOutput(videoDataOutput) // Create an object to output depth data. depthDataOutput = AVCaptureDepthDataOutput() depthDataOutput.isFilteringEnabled = false captureSession.addOutput(depthDataOutput) audioDeviceOutput = AVCaptureAudioDataOutput() audioDeviceOutput.setSampleBufferDelegate(self, queue: videoQueue) captureSession.addOutput(audioDeviceOutput) // Create an object to synchronize the delivery of depth and video data. outputVideoSync = AVCaptureDataOutputSynchronizer(dataOutputs: [depthDataOutput, videoDataOutput]) outputVideoSync.setDelegate(self, queue: videoQueue) // Enable camera intrinsics matrix delivery. guard let outputConnection = videoDataOutput.connection(with: .video) else { return } if outputConnection.isCameraIntrinsicMatrixDeliverySupported { outputConnection.isCameraIntrinsicMatrixDeliveryEnabled = true } } The top part of my delegate implementation is as follows: func dataOutputSynchronizer( _ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection ) { // Retrieve the synchronized depth and sample buffer container objects. guard let syncedDepthData = synchronizedDataCollection.synchronizedData(for: depthDataOutput) as? AVCaptureSynchronizedDepthData, let syncedVideoData = synchronizedDataCollection.synchronizedData(for: videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { if synchronizedDataCollection.synchronizedData(for: depthDataOutput) == nil { print("no depth data at time \(mach_absolute_time())") } if synchronizedDataCollection.synchronizedData(for: videoDataOutput) == nil { print("no video data at time \(mach_absolute_time())") } return } print("received depth data \(mach_absolute_time())") } As you can see, I'm console logging whenever depth data is not received. Note because I'm driving the video frames at 60 fps, its expected that I'll only receive depth data for every alternate video frame. Console output is posted as a follow up comment (because of the character limit). I edited some lines out for brevity. You'll see it started streaming correctly but after a while it stopped received both video and depth outputs (in some other runs, it works perfectly and in some other runs I receive no depth data whatsoever). One thing to note, I sometimes run quicktime mirroring to see the device screen to see what the app is displaying (so not sure if that's causing any interference - that said I don't see any system pressure changes either). Any help is most appreciated! Thanks.

I am currently working on a 2D pose estimator. I developed a PyTorch vision transformer based model with 17 joints in COCO format for the same and then converted it to CoreML using CoreML tools version 6.2. The model was trained on a custom dataset. However, upon running the converted model on iOS, I observed a significant drop in accuracy. You can see it in this video (https://youtu.be/EfGFrOZQGtU) that demonstrates the outputs of the PyTorch model (on the left) and the CoreML model (on the right). Could you please confirm if this drop in accuracy is expected and suggest any possible solutions to address this issue? Please note that all preprocessing and post-processing techniques remain consistent between the models. P.S. While converting I also got the following warning. : TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)): P.P.S. When we initialize the CoreML model on iOS 17.0, we get this error: Validation failure: Invalid Pool kernel width (13), must be [1-8] or 20. Validation failure: Invalid Pool kernel width (9), must be [1-8] or 20. Validation failure: Invalid Pool kernel width (13), must be [1-8] or 20. Validation failure: Invalid Pool kernel width (9), must be [1-8] or 20. Validation failure: Invalid Pool kernel width (13), must be [1-8] or 20. This neural network model does not have a parameter for requested key 'precisionRecallCurves'. Note: only updatable neural network models can provide parameter values and these values are only accessible in the context of an MLUpdateTask completion or progress handler.

I was hopeful that I would be able to record video from an external action camera such as the DJI Action Cam 3. The DJI action cam has a webcam mode that can be enabled as soon as one plugs it to a device, such as a macbook or an iPad. On the Macbook, the DJI shows up as an external camera for use in facetime or quicktime. But when I plug it into the iPad and use the AVCam sample from here, I notice that the camera preview comes up when the AVCam app is in photo mode but as soon as I switch the video mode, the preview layer hangs. No error message or AVCaptureSession errors that I could see. The same code works when using the Apple studio as an external camera. Curious if anyone has had any luck figuring this out? So near yet so far.

Hello, I'm finding that AVAssetWriter status is failing in the middle of recording frames obtained using RPScreenRecorder::startCapture. The failure seems to occur about a minute and half into a recording. The error code when asked from AVAssetWriter is -11847. It appears that error code is usually thrown when the app is backgrounded but in my case the app is never backgrounded. This error happens once every 2-3 runs on my iPhone 12 Pro Max. It's also reproducible on an iPhone 11. iOS version is 15.1.1. Any clues as to why this might be occurring? Thanks in advance. Note I'm using movieFragmentInterval on an mp4 file. I couldn't tell from the documentation if its only for QuickTime files or not. I'm setting up an AVAssetWriter like so. Code simplified for readability. let assetWriter = try? AVAssetWriter(outputURL: fileURL, fileType: AVFileType.mp4)! assetWriter.movieFragmentInterval = CMTimeMakeWithSeconds(60, preferredTimescale: 1000)       let preset = AVOutputSettingsAssistant(preset: .preset1280x720) var outputSettings = preset?.videoSettings outputSettings?[AVVideoScalingModeKey] = AVVideoScalingModeResizeAspect var compressionSettings = outputSettings?[AVVideoCompressionPropertiesKey] as? [String:Any] var bitRate:Float = 4.0 compressionSettings?[AVVideoAverageBitRateKey] = NSNumber(value:720.0 * 1280.0 * bitRate) outputSettings?[AVVideoCompressionPropertiesKey] = compressionSettings       videoInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: outputSettings) videoInput.expectsMediaDataInRealTime = true       assetWriter.add(videoInput)       let audioOutputSettings = preset?.audioSettings       audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioOutputSettings) audioInput.expectsMediaDataInRealTime = true;        assetWriter.add(audioInput) assetWriter.startWriting()

Hello, My device is an iPad Pro 2nd generation. Model number MXDC2LL/A. iOS 16 build 20A5283p. When creating an AVCaptureSession, isMultiTaskingCameraAccessSupported is returning false. Is this expected on this device? Is a newer device needed or is it an issue in this particular iOS 16 build? Thanks

My environment is as follows XCode version Version 14.0 beta 3 (14A5270f) iPad OS 16.0 beta 3 When querying for the devices property on a AVCapture.DiscoverySession, I get the following errors in the XCode console. captureSourceRemote_SetProperty signalled err=-16452 (kFigCaptureSourceError_SourceNotLocked) (Source must be locked for configuration to set properties) at FigCaptureSourceRemote.m:549 captureSourceRemote_SetProperty signalled err=-16452 (kFigCaptureSourceError_SourceNotLocked) (Source must be locked for configuration to set properties) at FigCaptureSourceRemote.m:549 The code is pretty simple and as follows: let videoDeviceDiscoverySession = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInWideAngleCamera, .builtInUltraWideCamera],                                              mediaType: .video, position: .unspecified) let devices = OFVideoDevice.videoDeviceDiscoverySession.devices Anyone know what this is about?

Hello, One of our customers is encountering an issue with video capture. The error is occurring when calling startRunning on an AVCaptureSession. A NSNotification.Name.AVCaptureSessionRuntimeError is triggered as soon as startRunning is called on the AVCaptureSession. The error returned is cryptic as found below. Can someone please tell me why this might be happening? Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-10851), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x283ababb0 {Error Domain-NSOSStatusErrorDomain Code=-10851 "(null)"}} This is happening every time the user tries to capture video. It's a rare issue in that 99.99% of our customers don't face this issue but this customer is facing this problem every time. Thanks

Hello, tldr; relaunch for background uploads isn't working as expected. I'm testing my code on iPad OS 14.6 and XCode 12.5.1. Background app refresh is ON on my device for my app. A background file upload is being setup with URLSession. The URL session is created as soon as the app starts up and not lazily created. See URL session configuration below. I have followed the test instructions as outlined here: https://developer.apple.com/forums/thread/14855, specifically launching my app not via Xcode but from the home screen and adding an exit() call after the app enters the background in the AppDelegate callback method applicationDidEnterBackground. The file transfer is initiated before the app enters the background. Here's the configuration code I'm using: let configuration = URLSessionConfiguration.background(withIdentifier: <unique_identifier>)     configuration.sessionSendsLaunchEvents = true     configuration.sharedContainerIdentifier = <container group identifier>     configuration.isDiscretionary = false     configuration.timeoutIntervalForResource = 3600 * 24 * 2 // wait for 2 days before timing out connections                urlSession = URLSession(configuration: configuration, delegate:self, delegateQueue: nil) The file in fact finishes uploading but the app isn't relaunched as one would expect. As soon as I re-open the app from the home screen, I get the requisite callbacks about the upload having completed properly. In the past when I have tested my code, the method AppDelegate:handleEventsForBackgroundURLSession used to be called after the upload finishes. That appears to no longer be the case. Any guidance/help will be hugely appreciated! Thanks Krishna