ARKit3: Body joints don't rotate

I use the code sample from "Capturing Body Motion in 3D"


https://developer.apple.com/documentation/arkit/capturing_body_motion_in_3d


I use this function


func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
        for anchor in anchors {
            guard let bodyAnchor = anchor as? ARBodyAnchor else { continue }
           
            // Update the position of the character anchor's position.
            let bodyPosition = simd_make_float3(bodyAnchor.transform.columns.3)
            let bodyOrientation = Transform(matrix: bodyAnchor.transform).rotation
            characterAnchor.position = bodyPosition + characterOffset
            /// Also copy over the rotation of the body anchor, because the skeleton's pose
            /// in the world is relative to the body anchor's rotation.
            characterAnchor.orientation = bodyOrientation

            if let character = character, character.parent == nil {
                // Attach the character to its anchor as soon as
                // 1. the body anchor was detected and
                // 2. the character was loaded.
                characterAnchor.addChild(character)
            }
        }
    }


Then I add the code to add or update the body joints:


            /// add or update joint spheres
            if let character = character {
                if jointSpheres.count <= character.jointNames.count, initalSpheresTransforms.count <= character.jointNames.count {                  
                    let remainingSpheresToAdd = initalSpheresTransforms.count - jointSpheres.count
                    if remainingSpheresToAdd > 0 {
                        if addSphereInterval < 5 {
                            addSphereInterval += 1
                        } else {
                            addSphereInterval = 0
                            let sphereToAddIndex = abs(remainingSpheresToAdd - initalSpheresTransforms.count)
                            let newSphere = CustomSphere(color: .blue, radius: 0.02)
                            newSphere.transform = initalSpheresTransforms[sphereToAddIndex]
                            sphereAnchor.addChild(newSphere, preservingWorldTransform: true)
                            jointSpheres.append(newSphere)
                        }

                    }
                    for  i in 0..                        let jointName = character.jointName(forPath: character.jointNames[i])
                        if let transform = bodyAnchor.skeleton.modelTransform(for: jointName) {
                            let position = bodyPosition + simd_make_float3(transform.columns.3)
                            jointSpheres[i].position = position
                            jointSpheres[i].orientation = bodyOrientation
                        }
                    }
               }

At first everything is okay.

When the person rotate 180° the roboter follows.


But the body joints stays in the same direction.

How can I set the correct position of the blue body joints after the rotation?

  • I am also running into this problem. Was a solution ever found to resolve this?

Add a Comment

Accepted Reply

Greetings,

I managed to get the body joints to follow the robot's rotation by rotating each of the modelTransform vectors by the bodyAnchor's rotation.

I.e change

let position = bodyPosition + simd_make_float3(transform.columns.3)

to

let position = bodyPosition + bodyOrientation.act(simd_make_float3(transform.columns.3))

Replies

Greetings,

I managed to get the body joints to follow the robot's rotation by rotating each of the modelTransform vectors by the bodyAnchor's rotation.

I.e change

let position = bodyPosition + simd_make_float3(transform.columns.3)

to

let position = bodyPosition + bodyOrientation.act(simd_make_float3(transform.columns.3))