AVCaptureDataOutputSynchronizer delegate call not containing depth information

Hey Everyone,


So I set up my AVCaptureDataOutputSynchronizer in both Swift and ObjC and it seems to be working in Swift but not ObjC so I'm really confused...


In Swift:


if session.canAddOutput(depthDataOutput) {
  session.addOutput(depthDataOutput)
  depthDataOutput.setDelegate(self, callbackQueue: dataOutputQueue)
}
outputSynchronizer = AVCaptureDataOutputSynchronizer(dataOutputs: [videoDataOutput, depthDataOutput])
outputSynchronizer!.setDelegate(self, queue: dataOutputQueue)


In ObjC:


if ( [_captureSession canAddOutput:_depthOutput] ) {
    [_captureSession addOutput:_depthOutput];
    [_depthOutput setDelegate:self callbackQueue:_videoDataOutputQueue];
}
_outputSynchronizer = [[AVCaptureDataOutputSynchronizer alloc] initWithDataOutputs:@[_videoOut, _depthOutput]];
[_outputSynchronizer setDelegate:self queue:_videoDataOutputQueue];


However, the depth info is missing in the delegate call from the output synchronizer in ObjC. I am able to see the depthDataOutput and the connection is there. When I do a print out of the connections for the outputSynchronizer I see the Depth and Video connections after I set them.


When I do three print outs of the properties at a break point in the delegate call for the output synchronizer in Swift I see:


AVCaptureSynchronizedDataCollection: 0x1c00110d0 { VDO[  ]: 590624033353208, DDO[  ]: 590624033353208, } (Latency:0.079288)
AVCaptureSynchronizedDataCollection: 0x1c400e430 { VDO[  ]: 590624100018333, DDO[  ]: 590624100018333, } (Latency:0.082406)
AVCaptureSynchronizedDataCollection: 0x1c400ec90 { VDO[  ]: 590624166683708, DDO[  ]: 590624166683708, } (Latency:0.083006)


Whereas in ObjC I see:

AVCaptureSynchronizedDataCollection: 0x1c4006cf0 { VDO[  ]: 590519171543071, DDO[--]:  , } (Latency:0.040787) AVCaptureDepthDataOutput: 0x1c002d560
AVCaptureSynchronizedDataCollection: 0x1c4007810 { VDO[  ]: 590519204872375, DDO[--]:  , } (Latency:0.042135) AVCaptureDepthDataOutput: 0x1c002d560
AVCaptureSynchronizedDataCollection: 0x1c4007760 { VDO[  ]: 590519238205083, DDO[--]:  , } (Latency:0.040983) AVCaptureDepthDataOutput: 0x1c002d560


So the Depth info is missing in the delegate call for some reason. So if anyone has any pointers to what I might be doing wrong I'd appreciate it!

Up vote post of Bobbyrob Down vote post of Bobbyrob
1.8k views
Posted by
Bobbyrob
Reply
Add a Comment

Accepted Reply

Is your depth data output's connection's active property YES or NO? If it's no, then your session is not set up to deliver depth. Either you've picked the wrong device, or you've picked the wrong session preset, or you've picked the wrong AVCaptureDevice activeFormat.

Posted by
bford
Up vote reply of bford Down vote reply of bford
Post marked as solved
Add a Comment

Replies

Is your depth data output's connection's active property YES or NO? If it's no, then your session is not set up to deliver depth. Either you've picked the wrong device, or you've picked the wrong session preset, or you've picked the wrong AVCaptureDevice activeFormat.

Posted by
bford
Up vote reply of bford Down vote reply of bford
Post marked as solved
Add a Comment

Thank you for the reply bford,


When I po videoDevice I get the Back Dual Camera:

AVCaptureFigVideoDevice: 0x102126410 [Back Dual Camera][com.apple.avfoundation.avcapturedevice.built-in_video:3]


When I po the _captureSession I get the right session:

AVCaptureSession: 0x1c400a210 [AVCaptureSessionPresetHigh]


When I po the _depthConnection it is inactive:

AVCaptureConnection: 0x1c400b2d0 [type:dpth][enabled:1][active:0]


But when I po the _videoConnection it's fine:

AVCaptureConnection: 0x1c400b260 [type:vide][enabled:1][active:1]


Yet, when I print out _outputSynchronizer.dataOutputs I see:

<__NSArrayI 0x1c002efc0>(
<AVCaptureVideoDataOutput: 0x1c403e980>,
<AVCaptureDepthDataOutput: 0x1c403b9a0>
)


So I'm really confused about why it seems to be configured correctly, yet it's not active and therefore not sending data to the delegate.


EDIT: I changed the preset to AVCaptureSessionPresetPhoto and it appears to be working, does the preset *have* to be AVCaptureSessionPresetPhoto in order for depth data to be returned?

Posted by
Bobbyrob
Up vote reply of Bobbyrob Down vote reply of Bobbyrob
Add a Comment

The AVCaptureSessionPresetHigh doesn't support depth. You need to pick a format that supports depth. One simple way to do it would be to set the sessionPreset AVCaptureSessionPresetPhoto. If you need a 16:9 video format, you'll need to search out the format you want and use AVCaptureDevice's setActiveFormat: API.

Posted by
bford
Up vote reply of bford Down vote reply of bford
Add a Comment

Thanks bford, I ended up landing on AVCaptureSessionPreset640x480 and just doubling the depth pixel buffer to mix it with the source stream myself.

Posted by
Bobbyrob
Up vote reply of Bobbyrob Down vote reply of Bobbyrob
Add a Comment