Detecting Tap Location on detected "PlaneAnchor"? Replacement for Raycast?

While PlaneAnchors are still not generated by the PlaneDetectionProvider in the simulator, I am still brainstorming how to detect a tap on one of the planes.

In an iOS ARKit application I could use a raycastQuery on existingPlaneGeometry to make an anchor with the raycast result's world transform.

I've not yet found the VisionOS replacement for this.

A possible hunch is that I need to install my own mesh-less PlaneModelEntities for each planeAnchor that's returned by the PlaneDetectionProvider. From there I can use a TapGesture targeted to those models? And then I could build a an WorldAnchor from the tap location on those entities.

Anyone have any ideas?

Up vote post of J0hn Down vote post of J0hn
634 views
Posted by
J0hn
Reply
Add a Comment

Accepted Reply

Hi, I've found that you can retrieve the RealityKit Scene from an entity, once added (subscribe to the event). Then you can use the Scene's Raycast methods, e.g: https://developer.apple.com/documentation/realitykit/scene/raycast(from:to:query😷 relativeto:) For that to work though you would need to create CollisionShapes for either the detected planes or the reconstructed world mesh geometry.

Posted by
arthurfromberlin
Up vote reply of arthurfromberlin Down vote reply of arthurfromberlin
Post marked as solved

  • I was hoping there was a way that did not involve manually creating and managing plane collision shapes as the PlaneAnchors change size and location.

    Also, I think if I did manage my own collision shapes, I could use a simpler TapGesture to detect the tap and 3D location of the tap instead of using a recast.

    —
    J0hn

  • I am going to mark this as the accepted answer. While I don’t like needing to place and manage my own Collision-enabled Entities, it enables both a SpatialTapGesture and raycast solution.

    —
    J0hn
Add a Comment

Replies

Hi, I've found that you can retrieve the RealityKit Scene from an entity, once added (subscribe to the event). Then you can use the Scene's Raycast methods, e.g: https://developer.apple.com/documentation/realitykit/scene/raycast(from:to:query😷 relativeto:) For that to work though you would need to create CollisionShapes for either the detected planes or the reconstructed world mesh geometry.

Posted by
arthurfromberlin
Up vote reply of arthurfromberlin Down vote reply of arthurfromberlin
Post marked as solved

  • I was hoping there was a way that did not involve manually creating and managing plane collision shapes as the PlaneAnchors change size and location.

    Also, I think if I did manage my own collision shapes, I could use a simpler TapGesture to detect the tap and 3D location of the tap instead of using a recast.

    —
    J0hn

  • I am going to mark this as the accepted answer. While I don’t like needing to place and manage my own Collision-enabled Entities, it enables both a SpatialTapGesture and raycast solution.

    —
    J0hn
Add a Comment