AVCaptureSession in RealityKit

Question

Created Dec ’20

Replies 3

Boosts 0

Views 1.4k

Participants 4

I have an AR app built with RealityKit. I want to capture a photo of the underlying scene in RAW (without the virtual objects). However, I am unable to find the AVCaptureSession (associated with the view?) in order to add the appropriate outputs.

Please help! The docs are not very complete on Realitykit. :(

Boost

Answer 1

IRONMANROMEO OP

Dec ’20

1

0

Answer 2

maxxfrazer OP

Dec ’20

You can get the CVPixelBuffer from arView.session.currentFrame?.capturedImage, which comes without any Entities shown.

From there you can convert to whatever format you like, Google something like "CVPixelBuffer to UIImage/CGImage/etc."

Code Block swiftif let capImg = self.arView.session.currentFrame?.capturedImage {
    let ciimg = CIImage(cvPixelBuffer: capImg)
    let myImage = UIImage(ciImage: ciimg)
    // save image or whatever
}

Not accounting for rotation in my given example, there's examples of how to do that with a quick search.

1

Answer 3

DTS Engineer OP

Apple

Dec ’20

"I want to capture a photo of the underlying scene in RAW"

There is no way for you to capture a RAW photo of the underlying scene (In RealityKit, SceneKit, or ARKit) while an ARSession is running, I recommend that you file an enhancement request using Feedback Assistant to request that feature.

2