Tracking different videos on multiple Images

Question

Created Dec ’21

Replies 1

Boosts 0

Participants 2

Hello ! I am currently trying to track multiple images with the ARKit - there should be a video tracked on every detected image like this: IMAGE A gets VIDEO a - IMAGE B gets VIDEO B.

I already managed to detect my two images and track a plane on them - but now I don't really know how to match the right video with the right image. I'm always facing some errors.

Here is my code so far:

import SceneKit
import ARKit
import AVFoundation
import SpriteKit

class ViewController: UIViewController, ARSCNViewDelegate {

  @IBOutlet var sceneView: ARSCNView!
   
   
  override func viewDidLoad() {
    super.viewDidLoad()
     
    // Set the view's delegate
    sceneView.delegate = self
    sceneView.autoenablesDefaultLighting = true
     
   
     
    // Show statistics such as fps and timing information
    sceneView.showsStatistics = true
     
    // Create a new scene
    let scene = SCNScene(named: "art.scnassets/shot1.scn")!
     
    // Set the scene to the view
    sceneView.scene = scene
  }
   
 override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
     
    // Create a session configuration
    let configuration = ARImageTrackingConfiguration()
     
    //Programm sagen nach welchen Bildern es suchen soll:
    if let trackingImages = ARReferenceImage.referenceImages(inGroupNamed: "Images", bundle: Bundle.main){
      configuration.trackingImages = trackingImages
      configuration.maximumNumberOfTrackedImages = 2
     
    }

    // Run the view's session
    sceneView.session.run(configuration)
  }
   
  override func viewWillDisappear(_ animated: Bool) {
    super.viewWillDisappear(animated)
     
    // Pause the view's session
    sceneView.session.pause()
  }
   
  //Node corresponding to a newly added anchor (Echtwelt Position und Orientation) Expects to return SCENE NODE
  func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
     
    let node = SCNNode()
    //Checken, ob gefundener Anker ein ARImageAnchor is
    if let imageAnchor = anchor as? ARImageAnchor{
    let size = imageAnchor.referenceImage.physicalSize //image Anchor gleich groß wie Echtgröße
       
      //PLANE
      let plane = SCNPlane(width: size.width, height: size.height) //Fläche erstellt mit Variable size (Echtgröße)
      plane.firstMaterial?.diffuse.contents = UIColor.white // Material von plane festgelegt = weiße Farbe
      let planeNode = SCNNode(geometry: plane)
      planeNode.eulerAngles.x = -.pi / 2
      node.addChildNode(planeNode)
       
      //VIDEOS
       
      let shot1URL = Bundle.main.url(forResource: "shot1", withExtension: "mp4")!
      let videoPlayerShot1 = AVPlayer(url: shot1URL)
      let shot2URL = Bundle.main.url(forResource: "shot2", withExtension: "mp4")!
      let videoPlayerShot2 = AVPlayer(url: shot2URL)
       
      let videoScene1 = SKScene(size: CGSize(width: 4096.0, height: 3072.0))
      let videoScene2 = SKScene(size: CGSize(width: 4630.0, height: 3274.0))
       
      let videoNode1 = SKVideoNode(avPlayer: videoPlayerShot1)
        videoNode1.position = CGPoint(x: videoScene1.size.width/2, y: videoScene1.size.height/2) // durch zwei um mittig zu sein
        videoNode1.size = videoScene1.size
        videoNode1.yScale = -1 // flips video damits nd kopfüber is
       
      videoScene1.addChild(videoNode1)
       
      let videoNode2 = SKVideoNode(avPlayer: videoPlayerShot2)
        videoNode2.position = CGPoint(x: videoScene2.size.width/2, y: videoScene2.size.height/2) // durch zwei um mittig zu sein
        videoNode2.size = videoScene2.size
        videoNode2.yScale = -1 // flips video damits nd kopfüber is
         
       
      videoScene2.addChild(videoNode2)
       

      if imageAnchor.referenceImage.name == "shot1Image" {
        let shot1 = planeNode.childNode(withName: "shot1", recursively: false)
        shot1!.geometry?.firstMaterial?.diffuse.contents = videoScene1
        videoNode1.play()
      } else{
        let shot2 = planeNode.childNode(withName: "shot2", recursively: false)
        shot2!.geometry?.firstMaterial?.diffuse.contents = videoScene2

        videoNode2.play()
      }
       
     
    }
     
    return node
     
  }

}

When I want to run the software on my IPhone it always says

com.apple.scenekit.scnview-renderer (10): Fatal error: Unexpectedly found nil while unwrapping an Optional value

so my expressions "shot1" and "shot2" are nil But I think that that's not the only thing I'm doing wrong right?

If would be glad if somebody could help me!!

Boost

Answer 1

robnotyou OP

Dec ’21

At a first glance, you are force-unwrapping "shot1" and "shot2", which is dangerous.
You should check that they are non-nil, before using them.

And perhaps add some simple error-logging, like this:

if imageAnchor.referenceImage.name == "shot1Image" {
    if let shot1 = planeNode.childNode(withName: "shot1", recursively: false) {
        shot1.geometry?.firstMaterial?.diffuse.contents = videoScene1
        videoNode1.play()
    } else {
        print("Error: couldn't get shot1")
    }
} else {
    if let shot2 = planeNode.childNode(withName: "shot2", recursively: false) {
        shot2.geometry?.firstMaterial?.diffuse.contents = videoScene2
        videoNode2.play()
    } else {
        print("Error: couldn't get shot1")
    }
}

Then if it turns out that shot1 or shot2 are nil, you need to figure out why.

0