Attach a Attachment to the hand VisionOS

Question

Created Jul ’24

Replies 2

Boosts 0

Participants 3

I am trying to attach a button to user's left hand. the position is tracked. the button stays above the user's left hand. but it doesn't face the user. or it doesn't even face where the wrist is pointing. this is the main code snippet:

            if (model.editWindowAdded) {
                let originalMatrix = model.originFromWristLeft
                
                
                let theattachment = attachments.entity(for: "sample")!
                entityDummy.addChild(theattachment)
                

                let testrotvalue = simd_quatf(real: 0.9906431, imag: SIMD3<Float>(-0.028681312, entityDummy.orientation.imag.y, 0.025926698))
                
                entityDummy.orientation = testrotvalue
                theattachment.position = [0, 0.1, 0]
                

                var timer = Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) {_ in
                    let originalMatrix = model.originFromWristLeft
                    
                    print(originalMatrix.columns.0.y)
                    
                    let testrotvalue = simd_quatf(real: 0.9906431, imag: SIMD3<Float>(-0.028681312,0.1, 0.025926698))

                    entityDummy.orientation = testrotvalue

                }
              
            }

Answered by snehdev in 795692022

Thank you. This is what I implemented as per my requirements. If anybody has any suggestions. feel free to add.

            if (model.editWindowAdded) {
                guard worldTracking.state == .running else { return }
                let deviceAnchor = worldTracking.queryDeviceAnchor(atTimestamp: CACurrentMediaTime())
                guard let deviceAnchor, deviceAnchor.isTracked else { return }

                let theattachment = attachments.entity(for: "sample")!
                entityDummy.addChild(theattachment)
                
                entityDummy.setTransformMatrix(model.handAnchorLeft.originFromAnchorTransform, relativeTo: nil)

                
                theattachment.position = [0, 0.1, 0]
                
                
                entityDummy.look(at: SIMD3<Float>(deviceAnchor.originFromAnchorTransform.columns.3.x,deviceAnchor.originFromAnchorTransform.columns.3.y,deviceAnchor.originFromAnchorTransform.columns.3.z), from: entityDummy.position, relativeTo: nil)
                
                
                let theyasix = entityDummy.transformMatrix(relativeTo: nil).gravityAligned.rotation
                entityDummy.setOrientation(theyasix, relativeTo: nil)
                
                let newOrientation1: Rotation3D
                let orientation2 = Rotation3D(entityDummy.orientation(relativeTo: nil))
                newOrientation1 = orientation2.rotated(by: .init(angle: .degrees(180), axis: .y))
                entityDummy.setOrientation(.init(newOrientation1), relativeTo: nil)
                
            }

Boost

Answer 1

Engineer OP

Apple

Jul ’24

it doesn't face the user

If you want it to always face the user, you can utilize the transform of the DeviceAnchor.originFromAnchorTransform and rotate your HandAnchor based transform to face that.

You can look at PlacementManager.updateUserFacingUIOrientations() in the "Tracking specific points in world space" Developer Sample:

https://developer.apple.com/documentation/visionos/tracking-points-in-world-space

it doesn't [...] face where the wrist is pointing

If you want the direction of the wrist, you can extract that information from the HandAnchor.originFromAnchorTransform or the wrist joint. It also helps to visualize the axis of each transform/joint to understand them better.

2

Answer 2

snehdev OP

Jul ’24

Accepted Answer

Thank you. This is what I implemented as per my requirements. If anybody has any suggestions. feel free to add.

            if (model.editWindowAdded) {
                guard worldTracking.state == .running else { return }
                let deviceAnchor = worldTracking.queryDeviceAnchor(atTimestamp: CACurrentMediaTime())
                guard let deviceAnchor, deviceAnchor.isTracked else { return }

                let theattachment = attachments.entity(for: "sample")!
                entityDummy.addChild(theattachment)
                
                entityDummy.setTransformMatrix(model.handAnchorLeft.originFromAnchorTransform, relativeTo: nil)

                
                theattachment.position = [0, 0.1, 0]
                
                
                entityDummy.look(at: SIMD3<Float>(deviceAnchor.originFromAnchorTransform.columns.3.x,deviceAnchor.originFromAnchorTransform.columns.3.y,deviceAnchor.originFromAnchorTransform.columns.3.z), from: entityDummy.position, relativeTo: nil)
                
                
                let theyasix = entityDummy.transformMatrix(relativeTo: nil).gravityAligned.rotation
                entityDummy.setOrientation(theyasix, relativeTo: nil)
                
                let newOrientation1: Rotation3D
                let orientation2 = Rotation3D(entityDummy.orientation(relativeTo: nil))
                newOrientation1 = orientation2.rotated(by: .init(angle: .degrees(180), axis: .y))
                entityDummy.setOrientation(.init(newOrientation1), relativeTo: nil)
                
            }

0