Walking an entity around an immersive space in visionOS like the window drag bar

Question

Created May ’24

Replies 2

Boosts 0

Participants 3

I'm trying to understand how Apple handles dragging windows around in an immersive space. 3d Gestures seem to be only half of the solution in that they are great if you're standing still and want to move the window an exaggerated amount around the environment, but if you then start walking while dragging, the amplified gesture sends the entity flying off into the distance. It seems they quickly transition from one coordinate system to another depending on if the user is physically moving. If you drag a window and start walking the movement suddenly matches your speed. When you stop moving, you can push and pull the windows around again like a super hero. Am I missing something obvious in how to copy this behavior? Hello world, which uses the 3d gesture has the same problem. You can move the world around but if you walk with it, it flies off. Are they tracking the head movement and if it's moved more than a certain amount it uses that offset instead? Is there anything out of the box that can do this before I try and hack my own solution?

Boost

Answer 1

DTS Engineer OP

Apple

May ’24

Hello @kallipigous,

Take a look at Transforming RealityKit entities using gestures, which implements functionality more along the lines of what you are looking for.

0

Answer 2

MarcoShapr OP

Jul ’24

Hi @kallipigous, I'm trying to achieve the same thing and I think I'm quite close but not there yet.

Basically when the drag gesture starts I use worldTrackingProvider.queryDeviceAnchor(atTimestamp: CACurrentMediaTime())?.originFromAnchorTransform to get the initial position of the device. Then at every gesture update I retrieve the same value, which I use to determine the device offset as let deviceAnchorOffset = deviceAnchorCurrentPosition - deviceAnchorStartPosition. If I understand correctly, what you are saying is that when deviceAnchorOffset goes above a certain threshold, instead of applying the offset that comes from DragGesture (which has a build-in multiplying factor that can't be controller), you apply the offset from the world space origin - which sounds the same as deviceAnchorOffset. Does this match your approach?

However I think this also assume your harm is stationary and just moving together in sync with your body, isn't it? Otherwise the object would still appear as slipping from your hands.

0