VisionOS: Simultaneous Drag & Rotate gestures

Question

Created May ’24

Replies 2

Boosts 1

Participants 3

I have been trying to replicate the entity transform functionality present in the magnificent app Museum That Never Was (https://apps.apple.com/us/app/the-museum-that-never-was/id6477230794) -- it allows you to simultaneously rotate, magnify and translate the entity, using gestures with both hands (as opposed to normal DragGesture() which is a one-handed gesture). I am able to rotate & magnify simultaneously but translating via drag does not activate while doing two-handed gestures. Any ideas? My setup is something like so:

Gestures:

    var drag: some Gesture {
        DragGesture()
            .targetedToEntity(where: QueryPredicate<Entity>.has(MyComponent.self))
            .onChanged { value in
                gestureTranslation = value.convert(value.translation3D, from: .local, to: .scene)
            }
            .onEnded { value in
                itemTranslation += gestureTranslation
                gestureTranslation = .init()
            }
    }
    
    var rotate: some Gesture {
        RotateGesture3D()
            .targetedToEntity(where: QueryPredicate<Entity>.has(MyComponent.self))
            .onChanged { value in
                gestureRotation = simd_quatf(value.rotation.quaternion).inverse
            }
            .onEnded { value in
                itemRotation = gestureRotation * itemRotation
                gestureRotation = .identity
            }
    }
    
    var magnify: some Gesture {
        MagnifyGesture()
            .targetedToEntity(where: QueryPredicate<Entity>.has(MyComponent.self))
            .onChanged { value in
                gestureScale = Float(value.magnification)
            }
            .onEnded { value in
                itemScale *= gestureScale
                gestureScale = 1.0
            }
    }

RealityView modifiiers:

        .simultaneousGesture(drag)
        .simultaneousGesture(rotate)
        .simultaneousGesture(magnify)

RealityView update block:

                entity.position = itemTranslation + gestureTranslation + exhibitDefaultPosition
                entity.orientation = gestureRotation * itemRotation
                entity.scaleAll(itemScale * gestureScale)

Boost

Answer 1

fportelacala OP

May ’24

I have a similar issue, any progress?

0

Answer 2

Vision Pro Engineer OP

Apple

Jul ’24

@matti777 The recommended way of doing a translation (moving of an object) is to use the standard single handed pinch and drag gesture as mentioned in Design for spatial input. The rotate and magnify gestures are designed to be performed with two hands.

You can also take a look at the sample code for transforming RealityKit entities using gestures to see the recommended way of implementing simultaneous gestures.

I would also like to mention that in the code snippets that you have shared, it is not necessary to update the transforms of the entity in the RealityView in the update block. You can do that in the gesture component that you have created as implemented in the sample code referenced above.

0