User Profile

I made the code as simple as possible: a single 5cm target square placed at location (0, 0, -1.5) I then tapped on target while standing in several locations, and the reported tap location was off along the Z axis by about 0.5m. Here were several tap locations (Z location is underlined): tap location: SIMD3(0.0067811073, 0.019996116, -1.1157947), name: target tap location: SIMD3(-0.00097223074, 0.019996116, -1.1036792), name: target tap location: SIMD3(0.0008024718, 0.019995179, -1.1074299), name: target tap location: SIMD3(-0.009804221, 0.019996116, -1.0694565), name: target tap location: SIMD3(-0.0037206858, 0.019995492, -1.0778457), name: target tap location: SIMD3(-0.009298846, 0.019996116, -1.0772702), name: target Here is the code to set up the RealityView: import SwiftUI import RealityKit import RealityKitContent struct ImmersiveView: View { @StateObject var model = MyModel() /// Spatial tap gesture that tells the model the tap location. var myTapGesture: some Gesture { SpatialTapGesture() .targetedToAnyEntity() .onEnded { event in let location3D = event.convert(event.location3D, from: .global, to: .scene) let entity = event.entity model.handleTap(location: location3D, entity: entity) } } var body: some View { RealityView { content in model.setupContentEntity(content: content) } .gesture(myTapGesture) } } Here is the model code: import Foundation import SwiftUI import RealityKit import RealityKitContent import ARKit import os.log @MainActor class MyModel: ObservableObject { private var realityViewContent: RealityViewContent? /// Capture RealityViewContent and create target /// /// - Parameter content: container for all RealityView content func setupContentEntity(content: RealityViewContent) { self.realityViewContent = content placeTargetObject() } /// Place a small red target at position 0, 0, -1.5 /// /// I will look at this position and tap my fingers. The tap location /// should be near the same position (0, 0, -1.5) func placeTargetObject() { guard let realityViewContent else { return } let width: Float = 0.05 let height: Float = 0.02 let x: Float = 0 let y: Float = 0 let z: Float = -1.5 // Create red target square let material = SimpleMaterial(color: .red, isMetallic: false) let mesh = MeshResource.generateBox(width: width, height: height, depth: width) let target = ModelEntity(mesh: mesh, materials: [material]) // Add collision and target component to make it tappable let shapeBox = ShapeResource.generateBox(width: width, height: height, depth: width) let collision = CollisionComponent(shapes: [shapeBox], isStatic: true) target.collision = collision target.components.set(InputTargetComponent()) // Set name, position, and add it to scene target.name = "target" target.setPosition(SIMD3<Float>(x,y + height/2, z), relativeTo: nil) realityViewContent.add(target) } /// Respond to the user tapping on an object by printing name of entity and tap location /// /// - Parameters: /// - location: location of tap gesture /// - entity: entity that was tapped func handleTap(location: SIMD3<Float>, entity: Entity) { os_log("tap location: \(location), name: \(entity.name, privacy: .public)") } } Example of the small red target:

I made some progress. When creating the SceneReconstructionProvider, specify the classifications mode. let sceneReconstruction = SceneReconstructionProvider(modes: [.classification]) Then the MeshAnchor.Geometry's classification property is set, and here are some example values count: 11342 description: GeometrySource(count: 11342, format: MTLVertexFormat(rawValue: 45)) format: 45 componentsPerVector: 1 offset: 0 stride: 1 So I am guessing the buffer contains a sequence values that map to the MeshAnchor.MeshClassification raw values. (Now I just need to figure out which MTLVertexFormat case has a raw value of 45 :-) Edit: uchar is type 45. So, the buffer contains a sequence of unsigned bytes.

Oops. It helps if I read the details. I'm rolling back to Beta 8

Note: Xcode > Settings > Platforms shows is there visionOS simulator is there. I'm going to delete all my Xcodes and try a fresh install.

I have found a workaround, but I don't know if this a good design or not. In "Deployment Info" in Xcode, I set iPhone/iPad Orientation to Portrait only. Then when I rotated the device to the side, the cube doesn't change sizes. (Note: I am getting the device's eulerAngles and converting them to a quaternion and applying that to the PerspectiveCamera)

I believe this is the expected behavior and consistent with RealityKit on the iPhone when attaching an anchor to a vertical surface. Attached is an old RealityKit test I did, where I anchored the object to the wall. In this case, the green cube is along the +X axis, the red cube is the +Y axis, and the blue cube is the +Z axis. If I recall correctly, when I attached things to the ceiling, the +Y axis pointed down. In general, I believe the +Y axis is the normal from the surface.

Figured it out! I needed to plug-in my phone and build for the actual device (as opposed to the simulator) Building for Any iOS Device (arm64) works too.

Success! In another thread someone mentioned installing older versions of Autodesk software, and that did the trick for me too. (Note: This is still with Monterey 12.2 not the 12.3 beta where I initially had problems) Here is the previous thread where I found the suggestion to try older versions of Autodesk software. https://developer.apple.com/forums/thread/661927