Detecting collisions between fingertip and world mesh

I'm using hand tracking to detect collisions between fingertips and entities that I have placed in the scene. I'm using the .mixed environment.

However, I want to detect when a fingertip touches a real-world object such as a wall.

No matter what I try, I can't get the collision to fire. I'm using the SceneReconstructionProvider to give me world meshes, which I use to create ModelEntity objects to which I add a CollisionComponent with the shape of the object.

I can render the meshes just fine, but nothing I do seems to allow collisions to work.

Surely this is possible, what am I missing?

Answered by rkeniger in 811486022

I found the solution here.

You need to allow static entities (i.e. the world mesh) to trigger collisions with kinematic entities (i.e. the fingertip entity). You do this by adding a physics simulation component to a parent entity (i.e. the root entity of your scene) and setting the appropriate collision option.

var physicsSim = PhysicsSimulationComponent()
physicsSim.collisionOptions.insert(.reportKinematicVsStatic)
contentEntity.components.set(physicsSim)
Accepted Answer

I found the solution here.

You need to allow static entities (i.e. the world mesh) to trigger collisions with kinematic entities (i.e. the fingertip entity). You do this by adding a physics simulation component to a parent entity (i.e. the root entity of your scene) and setting the appropriate collision option.

var physicsSim = PhysicsSimulationComponent()
physicsSim.collisionOptions.insert(.reportKinematicVsStatic)
contentEntity.components.set(physicsSim)

It's basically a ray-casting problem.

User-intention ray is:

  • Eye tracking
  • Device tracking
  • Hand tracking.

The source code of an example app of device tracking: https://github.com/CurvSurf/FindSurface-RealityKit-visionOS-Real-Time

Oh - I love this. Thank you for sharing @JoonAhn.

The source code below includes 'Eye tracking (spatial tap)', 'Device tracking', and 'Hand tracking'.

https://github.com/CurvSurf/FindSurface-RealityKit-visionOS-Response-to-Request

Detecting collisions between fingertip and world mesh
 
 
Q