SwiftUI and ARKit BodyTracking

Is anyone else seeing issues with ARkit Body Tracking and SwiftUI?


I am finding the the face tracking works fine if you have a view that appears with ARKit, does the tracking then goes away. This can consistently be done over and over by switching to the view and using it.


But I can't do the same with BodyTracking. If I have a view that is making use of BodyTracking then it only works if I show ONLY the view with the body tracking as the very first thing to appear when running the app. If I show any other views before switching to the one with the BodyTracking then it does not work. Also if I display that first view (which works fine) the switch to another view and back to the BodyTracking view then it no longer works.


This feels like a bug in ARKit, SwiftUI or iPadOS (don't have a phone to try it out).


Anyone else running into problems with SwiftUI and BodyTracking?


I get the following output when running my App (both when it works and when it doesn't). Feels to me like SwiftUI is causing issues with the tracking.


2019-09-30 17:05:24.543484+1300 BodyTracking [2080:711242] Metal GPU Frame Capture Enabled

2019-09-30 17:05:24.543752+1300 BodyTracking [2080:711242] Metal API Validation Enabled

2019-09-30 17:05:25.318275+1300 BodyTracking [2080:711242] Compiler failed to build request

2019-09-30 17:05:25.318408+1300 BodyTracking [2080:711242] [Graphics] makeRenderPipelineState failed [output of type ushort is not compatible with a MTLPixelFormatR16Float color attachement.].

2019-09-30 17:05:25.318437+1300 BodyTracking [2080:711242] [Graphics] makeRenderPipelineState failed.

2019-09-30 17:05:25.845971+1300 BodyTracking [2080:711618] [Technique] World tracking performance is being affected by resource constraints [3]

2019-09-30 17:05:25.846030+1300 BodyTracking [2080:711618] [Technique] VIO error callback: 113455.122269, 3, Frame time stamps are either out of order or repeating

Replies

Specifically the problem I am seeing is that the ARBodyAnchor is never found during the second attempt when using the View again.


Note that I am creating the ARView inside its own UIViewController itself, which is then wrapped in a UIViewControllerRepresentable for use in SwiftUI.


func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {

for anchor in anchors {

guard let bodyAnchor = anchor as? ARBodyAnchor else { continue }

//Never gets here after displaying the view the second time around.



Mainly what I am after here is acknowledgement from anyone else so that I know it's not just me. Then if possible advice for a workaround to get it going in SwiftUI. But my current plan is now to throw away my app completely and start again using the traditional app development approach (ie No SwiftUI at all, use storyboards and UIViewControllers the whole way). Which is what the example BodyTracking code does anyway, which makes me even more suspicious that it doesn't work properly yet in SwiftUI.

Did you end up fixing this issue? I'm having the exact same problem, and I am also considering switching back to the Interface Builder if I can't fix it.

It is possible to navigate to and from an ARView in SwiftUI and have body tracking work properly, if you are still unable to get body tracking to work when you return to your ARView then I suggest that you request technical support at https://developer.apple.com/support/technical/.