I'm trying to understand how to use the project(_:) function provided by ARView to convert 3D model coordinates to 2D screen coordinates, but am getting unexpected results.
Below is the default Augmented Reality App project, modified to have a single button that when tapped will place a circle over the center of the provided cube. However, when the button is pressed, the circle's position does not line up with the cube.
I've looked at the documentation for project(_:), but it doesn't give any details about how to convert a point from model coordinates to "the 3D world coordinate system of the scene". Is there better documentation somewhere on how to do this conversion?
// ContentView.swift
import SwiftUI
import RealityKit
class Coordinator {
var arView: ARView?
var anchor: AnchorEntity?
var model: Entity?
}
struct ContentView : View {
@State var coord = Coordinator()
@State var circlePos = CGPoint(x: -100, y: -100)
var body: some View {
ZStack {
ARViewContainer(coord: coord).edgesIgnoringSafeArea(.all)
VStack {
Spacer()
Circle()
.frame(width: 10, height: 10)
.foregroundColor(.red)
.position(circlePos)
Button(action: { showMarker() }, label: { Text("Place Marker") })
}
}
}
func showMarker() {
guard let arView = coord.arView else { return }
guard let model = coord.model else { return }
guard let anchor = coord.anchor else { return }
print("Model position is: \(model.position)")
// convert position into anchor's space
let modelPos = model.convert(position: model.position, to: anchor)
print("Converted position is: \(modelPos)")
// convert model locations to screen coordinates
circlePos = arView.project(modelPos) ?? CGPoint(x: -1, y: -1)
print("circle position is now \(circlePos)")
}
}
struct ARViewContainer: UIViewRepresentable {
var coord: Coordinator
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
coord.arView = arView
// Create a cube model
let mesh = MeshResource.generateBox(size: 0.1, cornerRadius: 0.005)
let material = SimpleMaterial(color: .gray, roughness: 0.15, isMetallic: true)
let model = ModelEntity(mesh: mesh, materials: [material])
coord.model = model
// Create horizontal plane anchor for the content
let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: SIMD2<Float>(0.2, 0.2)))
anchor.children.append(model)
coord.anchor = anchor
// Add the horizontal plane anchor to the scene
arView.scene.anchors.append(anchor)
return arView
}
func updateUIView(_ uiView: ARView, context: Context) {}
}
#Preview {
ContentView(coord: Coordinator())
}
To get a point in 3D world coordinates for use with project(_:), use Entity's convert(position:from:) instance method. The description of convert(position:from:)'s referenceEntity
says "Set this to nil to indicate world space.".
In the example above, the code:
// convert position into anchor's space
let modelPos = model.convert(position: model.position, to: anchor)
Should be changed to:
// convert position into world coordinate space
let modelPos = model.convert(position: model.position, to: nil)