Capture photo depth data in 32-bit

Question

Created Jun ’20

Replies 1

Boosts 0

Views 1.5k

Participants 2

Is it possible to set up a AVCaptureSession in a way that it will deliver 32-bit depth data (instead of 16-bit) during a photo capture?

I configured the AVCapturePhotoOutput and the AVCapturePhotoSettings to deliver depth data. And it works: my delegate receives a AVDepthData block… containing 16-bit depth data.

I tried setting the AVCaptureDevice's activeDepthDataFormat to a 32-bit format, but the format of the delivered AVDepthData is still only 16-bit—regardless of which format I set on the device.

For video capture using an AVCaptureDepthDataOutput this seem to work, just not for an AVCapturePhotoOutput.

Any hints are appreciated. 🙂

Answered by Media Engineer in 613222022

Hi Frank, what you've described is the expected behavior. Different engines are used for the disparity / depth generation in AVCaptureDepthDataOutput and AVCapturePhotoOutput. The photo output depth generation takes longer, but is more accurate. It is optimized to deliver depth (actually it's natively disparity) results that will be saved in image files (HEIC/JPEG). When saving depth data to image files, we always use disparity (it compresses better than depth), and translate the 16-bit floating point values to 8-bit fixed point values.

When using the depth data output, there's no requirement to save depth to files, so we allow its delivery as 16 or 32 bit floating point, disparity or depth. Natively, it is calculated as disparity (speaking of Dual / DualWide / Triple / TrueDepth cameras).

Boost

Answer 1

Media Engineer OP

Apple

Jun ’20

Accepted Answer

Hi Frank, what you've described is the expected behavior. Different engines are used for the disparity / depth generation in AVCaptureDepthDataOutput and AVCapturePhotoOutput. The photo output depth generation takes longer, but is more accurate. It is optimized to deliver depth (actually it's natively disparity) results that will be saved in image files (HEIC/JPEG). When saving depth data to image files, we always use disparity (it compresses better than depth), and translate the 16-bit floating point values to 8-bit fixed point values.

When using the depth data output, there's no requirement to save depth to files, so we allow its delivery as 16 or 32 bit floating point, disparity or depth. Natively, it is calculated as disparity (speaking of Dual / DualWide / Triple / TrueDepth cameras).

0