AVCam modified for Spatial Video captureing in WWDC24

I just follow the video and add the codes, but when I switch to spatial video capturing, the videoPreviewLayer shows black.

<<<< FigCaptureSessionRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSessionRemote.m:405) - (err=0)
<<<< FigCaptureSessionRemote >>>> captureSessionRemote_getObjectID signalled err=-16405 (kFigCaptureSessionError_ServerConnectionDied) (Server connection was lost) at FigCaptureSessionRemote.m:405
<<<< FigCaptureSessionRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSessionRemote.m:421) - (err=-16405)
<<<< FigCaptureSessionRemote >>>> Fig assert: "msg" at bail (FigCaptureSessionRemote.m:744) - (err=0)

Did I miss something?

Answered by Vision Pro Engineer in 791570022

Hello! Spatial video capture on iPhone 15 Pro requires that the capture connection's preferredVideoStabilizationMode is set to one of the cinematic modes before enabling spatial video capture. The cinematicExtendedEnhanced mode is the recommended mode to use.

Here's an expanded version of the code from the Build compelling spatial photo and video experiences WWDC video that shows how to set the stabilization mode.

class CaptureManager {
    var session: AVCaptureSession!
    var input: AVCaptureDeviceInput!
    var output: AVCaptureMovieFileOutput!

    func setupSession() throws -> Bool {
        session = AVCaptureSession()
        session.beginConfiguration()

        guard let videoDevice = AVCaptureDevice.default(
            .builtInDualWideCamera, for: .video, position: .back
        ) else { return false }

        var foundSpatialFormat = false
        for format in videoDevice.formats {
            if format.isSpatialVideoCaptureSupported {
                try videoDevice.lockForConfiguration()
                videoDevice.activeFormat = format
                videoDevice.unlockForConfiguration()
                foundSpatialFormat = true
                break
            }
        }
        guard foundSpatialFormat else { return false }

        let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)
        guard session.canAddInput(videoDeviceInput) else { return false }
        session.addInput(videoDeviceInput)
        input = videoDeviceInput

        let movieFileOutput = AVCaptureMovieFileOutput()
        guard session.canAddOutput(movieFileOutput) else { return false }
        session.addOutput(movieFileOutput)
        output = movieFileOutput

        guard let connection = output.connection(with: .video) else { return false }
        guard connection.isVideoStabilizationSupported else { return false }
        connection.preferredVideoStabilizationMode = .cinematicExtendedEnhanced

        guard movieFileOutput.isSpatialVideoCaptureSupported else { return false }
        movieFileOutput.isSpatialVideoCaptureEnabled = true

        session.commitConfiguration()
        session.startRunning()
        return true
    }

}
Accepted Answer

Hello! Spatial video capture on iPhone 15 Pro requires that the capture connection's preferredVideoStabilizationMode is set to one of the cinematic modes before enabling spatial video capture. The cinematicExtendedEnhanced mode is the recommended mode to use.

Here's an expanded version of the code from the Build compelling spatial photo and video experiences WWDC video that shows how to set the stabilization mode.

class CaptureManager {
    var session: AVCaptureSession!
    var input: AVCaptureDeviceInput!
    var output: AVCaptureMovieFileOutput!

    func setupSession() throws -> Bool {
        session = AVCaptureSession()
        session.beginConfiguration()

        guard let videoDevice = AVCaptureDevice.default(
            .builtInDualWideCamera, for: .video, position: .back
        ) else { return false }

        var foundSpatialFormat = false
        for format in videoDevice.formats {
            if format.isSpatialVideoCaptureSupported {
                try videoDevice.lockForConfiguration()
                videoDevice.activeFormat = format
                videoDevice.unlockForConfiguration()
                foundSpatialFormat = true
                break
            }
        }
        guard foundSpatialFormat else { return false }

        let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)
        guard session.canAddInput(videoDeviceInput) else { return false }
        session.addInput(videoDeviceInput)
        input = videoDeviceInput

        let movieFileOutput = AVCaptureMovieFileOutput()
        guard session.canAddOutput(movieFileOutput) else { return false }
        session.addOutput(movieFileOutput)
        output = movieFileOutput

        guard let connection = output.connection(with: .video) else { return false }
        guard connection.isVideoStabilizationSupported else { return false }
        connection.preferredVideoStabilizationMode = .cinematicExtendedEnhanced

        guard movieFileOutput.isSpatialVideoCaptureSupported else { return false }
        movieFileOutput.isSpatialVideoCaptureEnabled = true

        session.commitConfiguration()
        session.startRunning()
        return true
    }

}

If I modified codes as you provide, it seems to be able to record the spatial video, but the camera preview will stop. After pressing the record button and startRecording, it seems running but no record leave.

Maybe we need some more detail for the preview using dualwidth camera

Sorry and where is "foundSpatialFormat" used for?

Oh... I write it with other sample it works. Thank you.

AVCam modified for Spatial Video captureing in WWDC24
 
 
Q