Using SpatialTapGesture location with AnchorEntity hierarchies on visionOS

Yeah, I get the same behavior. There are a few posts here of others finding the same behavior too.

I think you can only get location information relative to the AnchorEntity’s origin from that entity and its children.

In addition, AnchorEntities and their children cannot behave in physics based interactions outside of that anchors hierarchy and that includes ray casting.

I’ve been endeavoring to never use an AnchorEntoty because of these limitations. Thankfully the plane provider that works on device doesn’t require those entities or else I would be sunk.

J0hn is correct, as mentioned here, "AnchorEntity's exist in an independent coordinate space and an independent physics simulation."

Please file an enhancement request using Feedback Assistant if you are unable to construct your app without AnchorEntity, or if you feel that it would be improved with some change to AnchorEntity.

thanks @J0hn and @gchiste - I'm relieved it isn't just me :)

just to make sure i'm understanding :)... I'm not (yet) wanting to use AnchorEntities with physics sim, this is just when using spatial gesture modifiers. are those modifiers using the physics sim under the hood for calculating the tap location (i.e. raycasting as j0hn mentioned)?

if i can get the tap location in a coordinate space useable by the anchored entities then i think that'd be fine?

the SpatialTapGesture modifier is correctly called when i tap on an anchored entity & reports the correct entity in the event, but I can't figure out how to use the event's location3D to create a new entity that will sit at that tap location under the same AnchorEntity, so in the same coordinate space as the tapped entity.

i really want to use the plane detector, but currently only have the simulator to work with.

if this is as currently designed then i'll file a request.. just wanted to double check first :)

thanks!

ahh thanks @gchiste - i was thinking that was the case.

so.. and forgive me because i'm sure i'm missing something straightforward here.. given i want to get the location of a tap on an entity that is a child of an anchor, and create a new entity at that position within the same hierarchy under the anchor... the places where coordinate spaces come into play afaik are:

SpatialTapGesture()

which can take a coordinateSpace param of either local, global or named

value.convert(value.location3D, from: .local, to: .scene)

where from can be local, global or named. and to can be scene or an entity

and finally the code which creates the entity, sets it's location, and adds it to an entity in the anchor hierarchy

        tap.position = location3D
        value.entity.addChild(tap, preservingWorldTransform: true)

any idea what the combination of params, coordinate spaces etc i should be using here?

happy new year! :) and exciting news today about the feb 2nd launch!!

i just wanted to come back to this as i am stumped.. i'm sure i'm doing something really dumb...

this is my test gesture handler.. anchor is an AnchorEntity with a plane target. the grey sphere is a child of the anchor at the anchor's origin, and has a CollisionComponent and an InputTargetComponent.

        .gesture(SpatialTapGesture().targetedToAnyEntity()
            .onChanged({ value in
            })
            .onEnded({ value in
                let material = SimpleMaterial(color: .systemPink, roughness: 0.1, isMetallic: true)
                
                let location3D = value.convert(value.location3D, from: .local, to: anchor) // i've tried .scene.. specifying the anchor seems to be slightly closer results. i've also tried .global instead of .local for the from
                let tap = createSphereEntity(0.01, material: material, interactable: false)
                anchor.addChild(tap, preservingWorldTransform: false)
                tap.setPosition(location3D, relativeTo: nil)
             })
        )

i've attached an image of the result i get. as you can see, the "tap" entities that are created are offset from the anchor. i've tried so many combinations of coordinate spaces, preserving world transforms, changing the order of things etc...

if the object is not anchored

the following code works great for non anchored entities:

        .gesture(SpatialTapGesture().targetedToAnyEntity()
            .onChanged({ value in
            })
            .onEnded({ value in
                let material = SimpleMaterial(color: .systemPink, roughness: 0.1, isMetallic: true)
                
                let location3D = value.convert(value.location3D, from: .local, to: .scene)
                let tap = createSphereEntity(0.01, material: material, interactable: false)
                value.entity.addChild(tap, preservingWorldTransform: false)
                tap.setPosition(location3D, relativeTo: nil)
             })

see the screenshot with the yellow shows the expect results

any ideas what i'm doing wrong? :)

d'oh, ok, having thought about it some more i think i see the root of my misunderstanding :) i thought that since i only wanted to work in the anchor's coordinate space it shouldn't be affected by the privacy preserving.. but of course, as it currently stands if that were possible, it'd also be possible to figure out the world space positions of anchors and their children.

to be honest, the only reason i want plane AnchorEntities is that they're supported in editor and the ARKit planes api is not.. hoping maybe maybe that will change 🤞