Node placed in didAdd moves with camera on smaller target

I'll apologize in advance in case this is either a dumb question or a poorly worded one. I'm learning to use ARKit via online resources which I'm finding to be a bit thin.

I have an app which scans still images and places a cube on one that it recognizes. My first version of this app added the node in didAdd, which is how most sample code does it online:

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
    guard let imageAnchor = anchor as? ARImageAnchor else { return }
    updateQueue.async {
      let cube = self.createCubeNode() // returns an SCNNode with SCNBox geometry
      let position = SCNVector3(x: imageAnchor.transform.columns.3.x,
                                y: imageAnchor.transform.columns.3.y,
                                z: imageAnchor.transform.columns.3.z)
      cube.worldPosition = position
      self.sceneView.scene.rootNode.addChildNode(cube)
    }
  }

This worked as expected if the image was displayed on my monitor or on a TV, but when the image was on another iPhone, the cube wandered around a bit as I moved th scanning phone from side to side.

I found many creative solutions to this in StackOverflow, none of which worked for me. I finally figured out, mostly by accident, that if I instead create the node in nodeFor and let ARKit add it, this does not happen.

  func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
    guard let _ = anchor as? ARImageAnchor else { return nil }
    return self.createCubeNode()
  }

This works, since it put the cube in the center of the image which is where I wanted it anyway, but I don't understand why it doesn't work properly from didAdd. Nor do I understand why it only happens when the displayed image is small.

Thanks in advance!

Answered by Vision Pro Engineer in 744611022

In renderer(_:nodeFor:) ARKit creates a node and automatically keeps it in sync as the anchor changes. So when you move the iPhone that displays the reference image, the cube will remain attached to the phone.

renderer(_:didAdd:for:) is called once when the image is initially detected. With the line

self.sceneView.scene.rootNode.addChildNode(cube)

you are adding the cube to the scene's root node, which means it remains unaware of future updates of the image anchor. You could instead add it as a child to the node that was created for the anchor. Then it will be updated automatically:

node.addChildNode(cube)

You don't even have to retrieve the image anchor's world position in that case, because node already has the correct position.

This is the preferred way of associating content with an anchor. If you still wanted to see the cube updated after adding it to the root node, you would need to also implement renderer(_:didUpdate:for:) and set the cube's transform to that of the image anchor.

I've been playing around with this a bit more.

It turns out that I can use use didAdd instead of nodeFor if I don't set a position for the cube, and if I add it to the passed in node instead of the root node. Any other variation has the cube moving around to some degree.

It also only holds still if I use an ARImageTrackingConfiguration. If I use a ARWorldTrackingConfiguration instead, which I believe should also work, the cube moves no matter what else I do.

Why?

Accepted Answer

In renderer(_:nodeFor:) ARKit creates a node and automatically keeps it in sync as the anchor changes. So when you move the iPhone that displays the reference image, the cube will remain attached to the phone.

renderer(_:didAdd:for:) is called once when the image is initially detected. With the line

self.sceneView.scene.rootNode.addChildNode(cube)

you are adding the cube to the scene's root node, which means it remains unaware of future updates of the image anchor. You could instead add it as a child to the node that was created for the anchor. Then it will be updated automatically:

node.addChildNode(cube)

You don't even have to retrieve the image anchor's world position in that case, because node already has the correct position.

This is the preferred way of associating content with an anchor. If you still wanted to see the cube updated after adding it to the root node, you would need to also implement renderer(_:didUpdate:for:) and set the cube's transform to that of the image anchor.

Node placed in didAdd moves with camera on smaller target
 
 
Q