Cannot assign AVCaptureDevice to SCNMaterialProperty.contents

I want to apply a SCNTechnique pipeline to the camera feed. To achieve this, I want to bring the camera input into the SceneKit world.

The perfects API seems to be:

let captureDevice = …
scnScene.background.contents = captureDevice

This is demonstrated in "SceneKit: What's New" (WWDC17) (at 44m19s) and is mentioned in the documentation of SCNMaterialProperty's contents.

Instead of showing camera feed, it crashes with these messages:

*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[AVCaptureVideoDataOutput setVideoSettings:] Unsupported pixel format type - use -availableVideoCVPixelFormatTypes'
*** First throw call stack:
(0x18993c7cc <REDACTED> 0x211e18488)
libc++abi: terminating due to uncaught exception of type NSException

Please advise.

STEPS TO REPRODUCE

  • Create a new Xcode project, starting from the SceneKit game template.
  • Add Info.plist entry for NSCameraUsageDescription.
  • Add a capture device property to GameViewController:
    class GameViewController: UIViewController {
    let captureDevice = AVCaptureDevice.default(for: .video)
    
  • Set the background contents:
    scene.background.contents = captureDevice
    
  • Run the app on device.

PLATFORM AND VERSION

iOS Development environment: Xcode 16.1, macOS 15.0.1. Run-time configuration: iOS 18.1

Answered by DTS Engineer in 813726022

Hello @CloakedEddy,

Thank you for reaching out about this!

Please file a bug report for this issue using Feedback Assistant.

Best regards,

Greg

Hello @CloakedEddy,

Thank you for reaching out about this!

Please file a bug report for this issue using Feedback Assistant.

Best regards,

Greg

Cannot assign AVCaptureDevice to SCNMaterialProperty.contents
 
 
Q