RealityKit, ImageAnchor, How to keep tracking outside of ImageAnchor

Hello,

There are a couple of things happening here:

1. ARView is automatically building and running an ARConfiguration based on the content in the scene. For example, in the default Experience.rcproject, with the anchor type set to "Image", ARView will build an ARWorldTrackingConfiguration along with a detectionImages set once the rcproject's content has been added to the Scene. You do not need to disable automaticallyConfigureSession to get the behavior you'd like.

2. When an AnchorEntity is initialized with an image target (which the rcproject does automatically for you), the behavior is that, when the ARImageAnchor that the anchor entity has associated with is not currently being tracked, the AnchorEntity will hide itself (i.e. it will not render or participate in collisions).

So, the solution that you are looking for is to make use of ARSessionDelegate's methods to manually update the position of a general AnchorEntity (i.e. one which does not have a target type specified), and add the model entity that you'd like to see track the image as a child of the AnchorEntity. In code, this could look something like this:

    let boxAnchor = try! Experience.loadBox()
    var imageAnchorToEntity: [ARImageAnchor: AnchorEntity] = [:]
   
    override func viewDidLoad() {
        super.viewDidLoad()
       
        arView.scene.addAnchor(boxAnchor)
        arView.session.delegate = self
    }
   
    func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
        anchors.compactMap { $0 as? ARImageAnchor }.forEach {
            let anchorEntity = AnchorEntity()
            let modelEntity = boxAnchor.steelBox!
            anchorEntity.addChild(modelEntity)
            arView.scene.addAnchor(anchorEntity)
            anchorEntity.transform.matrix = $0.transform
            imageAnchorToEntity[$0] = anchorEntity
        }
    }

    func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
        anchors.compactMap { $0 as? ARImageAnchor }.forEach {
            let anchorEntity = imageAnchorToEntity[$0]
            anchorEntity?.transform.matrix = $0.transform
        }
    }