RayCasting to Surface Returns Inconsistent Results?

On Xcode 15.1.0b2 when rayacsting to a collision surface, there appears to be a tendency for the collisions to be inconsistent.

Here are my results. Green cylinders are hits, and red cylinders are raycasts that returned no collision results.

NOTE: This raycast is triggered by a tap gesture recognizer registering on the cube... so it's weird to me that the tap would work, but the raycast not collide with anything.

Is this something that just performs poorly in the simulator?

My RayCasting command is:

guard let pose = self.arSessionController.worldTracking.queryDeviceAnchor(atTimestamp: CACurrentMediaTime()) else {
            print("FAILED TO GET POSITION")
            return
        }
        let transform = Transform(matrix: pose.originFromAnchorTransform)
        let locationOfDevice = transform.translation
        
        let raycastResult = scene.raycast(from: locationOfDevice, to: destination, relativeTo: nil)

where destination is retrieved in a tap gesture handler via:

 let worldPosition: SIMD3<Float> = value.convert(value.location3D, from: .local, to: .scene)

Any findings would be appreciated.