How to transform Vision body tracking coordinates

Question

Created Jun ’20

Replies 3

Boosts 0

Views 1.5k

Participants 1

I am using the new Vision VNDetectHumanBodyPoseRequest from Apple, but I am not sure why my mapped coordinates are incorrect with the front-facing camera.

My drawing code is correct as I can see the nodes, but they are in the completely wrong position. I tried setting different orientations in the VNImage request handler:

Code Block VNImageRequestHandler(cmSampleBuffer: sampleBuffer, orientation: .left)

The file is attached.

CameraViewController.swift

Thanks for the help!

Answered by LemonSpike in 618331022

I figured it out! The body needs to be further away from the camera: tracking performs very well for distances greater than 5m apart, even in horizontal movements or obscured joint positions.

Boost

Answer 1

LemonSpike OP

Jun ’20

I fixed the skeleton, it's now visible, but there is considerable lag and jerkiness in the movement.

Code Block captureConnection?.videoOrientation = .portrait
...
let requestHandler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer, orientation: .down)
...
var recognizedPoints = recFacePoints.merging(recTorsoPoints) { (current, _) in current }
recognizedPoints = recognizedPoints.merging(recLeftArmPoints) { (current, _) in current }
recognizedPoints = recognizedPoints.merging(recRightArmPoints) { (current, _) in current }
recognizedPoints = recognizedPoints.merging(recLeftLegPoints) { (current, _) in current }
recognizedPoints = recognizedPoints.merging(recRightLegPoints) { (current, _) in current }

were the changes I made

0

Answer 2

LemonSpike OP

Jun ’20

Are there any ways to reduce this jerkiness and improve performance? Thanks in advance.

0

Answer 3

LemonSpike OP

Jul ’20

Accepted Answer

I figured it out! The body needs to be further away from the camera: tracking performs very well for distances greater than 5m apart, even in horizontal movements or obscured joint positions.

1