TrueDepth Camera - Body Tracking

Hi Guys,

The specs of the TrueDepth Camera on the iPhoneX say; "Body and face detection". I've found the documentation on Face tracking (ARFaceTrackingConfiguration). Is anyone aware of an SDK/Framework for Body tracking using the front facing TrueDepth Camera? (i.e something along the lines of the original Kinect api that does skeletal tracking https://msdn.microsoft.com/en-us/library/dn799273.aspx ) Many Thanks

Replies

I’m also working on this issue.

Did you ever succeed with this? I'm very interested in doing this. I'd love to know if you had any luck.

Any update on this? Is there any way to get ARBodyTrackingConfiguration to work with the front facing camera?
ARBodyTracking is only supported on the rear camera (at the time of this writing, in case anyone stumbles across this later, the current release is iOS/iPadOS 14, which indicates the back camera feed is required for an ARBodyTrackingConfiguration). If your app supports iOS/iPadOS 14 or higher, you could consider leveraging the Vision framework, which is capable of detecting body poses and joints, on both the front and rear camera. The Detecting Human Body Poses in Images documentation details much about this technology, including a list of supported joints. While ARBodyTracking supports recognizing more joints and applying the transforms to a 3D model, for a motion capture/puppet-like effect, depending on your use case, the Vision-base body pose recognition could suffice your needs.

We can get true depth map from AV Foundation using front camera but its not possible to convert screen points to 3D coordinates in AV. To track body with front camera we have to use AR 2D body tracking or AR Face tracking provided by Apple itself. We can still use Vision Human body pose detection (Vision 3D body tracking) but it seems not to be much stable and accurate on live camera feed