+1 I just spent a lot of time writing my first answer on these forums to a question only to get the message "The maximum number of characters has been reached. Please refine your answer." when clicking the submit button. How about saying what the character limit is and how much I went over?!? The live preview looks fine and doesn't show anything about the length of the post, which would have been a great place to show a warning. FYI, the error message does not provide any workaround suggestions like moving code to attachments, but I guess I'll try that or maybe post multiple answers ... Since this was posted 10 months ago, it doesn't seem like the bug report helped, but I'll file a new one anyway
Post
Replies
Boosts
Views
Activity
I'm having the same problem on an 11" 2020 iPad Pro. This is what I've found out so far. Perhaps someone from Apple can confirm that this device should be capable of producing depth data from the rear camera.
After doing some debugging, I determined that this iPad does not report having AVCaptureDevice .builtInDualCamera. It does have .builtInDualWideCamera, however that AVCaptureDevice type is not showing depth support.
This is the relevant section of the sample code in CameraViewModel.swift starting at line 550:
/// This method checks for a depth-capable dual rear camera and, if found, returns an `AVCaptureDevice`.
private func getVideoDeviceForPhotogrammetry() throws -> AVCaptureDevice {
var defaultVideoDevice: AVCaptureDevice?
// Specify dual camera to get access to depth data.
if let dualCameraDevice = AVCaptureDevice.default(.builtInDualCamera, for: .video,
position: .back) {
logger.log(">>> Got back dual camera!")
defaultVideoDevice = dualCameraDevice
} else if let dualWideCameraDevice = AVCaptureDevice.default(.builtInDualWideCamera,
for: .video,
position: .back) {
logger.log(">>> Got back dual wide camera!")
defaultVideoDevice = dualWideCameraDevice
} else if let backWideCameraDevice = AVCaptureDevice.default(.builtInWideAngleCamera,
for: .video,
position: .back) {
logger.log(">>> Can't find a depth-capable camera: using wide back camera!")
defaultVideoDevice = backWideCameraDevice
}
guard let videoDevice = defaultVideoDevice else {
logger.error("Back video device is unavailable.")
throw SessionSetupError.configurationFailed
}
return videoDevice
}
You'll see in the console output when running with your iPad attached that it prints >>> Got back dual wide camera!, having passed over the first choice of .builtInDualCamera. The way the code is written, it looks like that wide dual camera would also support depth because only the next else if selection (.builtInWideAngleCamera) specifically says that it can't find a depth-capable camera. However, I found that the dual wide camera does not support depth information by adding this code:
let discoverySession = AVCaptureDevice.DiscoverySession(deviceTypes:
[.builtInDualCamera, .builtInDualWideCamera, .builtInUltraWideCamera, .builtInTelephotoCamera, .builtInWideAngleCamera, .builtInTrueDepthCamera, .builtInTripleCamera],
mediaType: .video, position: .unspecified)
for device in discoverySession.devices {
print("\(device) supports \(device.activeFormat.supportedDepthDataFormats)")
}
Console Output:
<AVCaptureFigVideoDevice: 0x104915b50 [Back Dual Wide Camera][com.apple.avfoundation.avcapturedevice.built-in_video:6]> supports []
<AVCaptureFigVideoDevice: 0x104915480 [Back Ultra Wide Camera][com.apple.avfoundation.avcapturedevice.built-in_video:5]> supports []
<AVCaptureFigVideoDevice: 0x104914810 [Back Camera][com.apple.avfoundation.avcapturedevice.built-in_video:0]> supports []
<AVCaptureFigVideoDevice: 0x104916140 [Front Camera][com.apple.avfoundation.avcapturedevice.built-in_video:1]> supports []
<AVCaptureFigVideoDevice: 0x104916750 [Front TrueDepth Camera][com.apple.avfoundation.avcapturedevice.built-in_video:4]> supports ['dpth'/'hdis' 160x 90, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'fdis' 160x 90, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'hdep' 160x 90, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'fdep' 160x 90, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'hdis' 320x 180, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'fdis' 320x 180, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'hdep' 320x 180, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'fdep' 320x 180, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'hdis' 640x 360, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'fdis' 640x 360, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'hdep' 640x 360, { 2- 30 fps}, HRSI: 640x 360, fov:61.161, 'dpth'/'fdep' 640x 360, { 2- 30 fps}, HRSI: 640x 360, fov:61.161]
Here is a description of those two device types from https://developer.apple.com/documentation/avfoundation/avcapturedevice/devicetype
static let builtInDualCamera: AVCaptureDevice.DeviceType
// A combination of wide-angle and telephoto cameras that creates a capture device.
static let builtInDualWideCamera: AVCaptureDevice.DeviceType
// A device that consists of two cameras of fixed focal length, one ultrawide angle and one wide angle.
Based on that description and the fact that this iPad Pro only has two rear cameras, it must be lacking the telephoto camera while having a wide and ultra-wide camera. The iPhone 12 Pro has 3 cameras, so I'm guessing it has the missing telephoto lens. I'm buying one tomorrow and I'll report if I can get this demo working properly. I suspect that this demo was written for and tested on an iPhone. In some ways that makes sense as the iPad gets pretty heavy after taking hundreds of pictures but it would be frustrating if the iPad with LIDAR doesn't support capturing rear depth data as many people, myself included, bought it specifically to use it for scanning. I'm hoping that .builtInDualWideCamera not reporting depth support is a bug!
Apple forums said my post was too long ?? so this is (1/2)
(2/2) since this part of the answer made my first response too long... (note: the forum error message only shows up when trying to submit your answer, not in the live preview, and it doesn't say how much you are over. frustrating!)
There is one way to get depth images with this iPad and that is to use the front camera (.builtInTrueDepthCamera), which you can do by altering the code from my other answer to search for the true depth camera:
if let dualCameraDevice = AVCaptureDevice.default(.builtInDualCamera, for: .video,
position: .back) {
logger.log(">>> Got back dual camera!")
defaultVideoDevice = dualCameraDevice
} else if let trueDepthDevice = AVCaptureDevice.default(.builtInTrueDepthCamera, for: .video, position: .unspecified) {
logger.log(">>> Got true depth camera!")
defaultVideoDevice = trueDepthDevice
} else if let dualWideCameraDevice = AVCaptureDevice.default(.builtInDualWideCamera,
for: .video,
position: .back) {
logger.log(">>> Got back dual wide camera!")
// Note: this does not have depth data. Why?!?
defaultVideoDevice = dualWideCameraDevice
} else if let backWideCameraDevice = AVCaptureDevice.default(.builtInWideAngleCamera,
for: .video,
position: .back) {
logger.log(">>> Can't find a depth-capable camera: using wide back camera!")
defaultVideoDevice = backWideCameraDevice
}
You'll notice when running the app that the yellow ! changes to a green check mark and depth data is recorded to the image. Of course this is a completely impractical way to scan an object because it's hard to see the screen without getting in the photo! Also, unfortunately, the resolution of the depth image (640x480) is much smaller than the color image (3088x2316), but hopefully that won't be the case with the dual rear cameras.
My bug number is FB9189559
I can confirm that it doesn't work on the 13" MBP that I have (2020, Intel i5, 16gb RAM). I get this Error creating session: cantCreateSession("Native session create failed: CPGReturn(rawValue: -11)")
Based on the slides in the presentation, it should work on the newest MBP with the M1 chip or on the 16" Intels with a 4GB AMD video card 16gb RAM. Interestingly, it also works on the macbook air with the M1 chip and on the M1 mac mini.
I see that Apple has a 14 day return period, so I'm going to buy an 8gb one tomorrow and I'll report my results. If it doesn't fit my needs, at least it will be easy to return and upgrade to 16gb :-)