AVCaptureSession in RealityKit

I have an AR app built with RealityKit. I want to capture a photo of the underlying scene in RAW (without the virtual objects). However, I am unable to find the AVCaptureSession (associated with the view?) in order to add the appropriate outputs.

Please help! The docs are not very complete on Realitykit. :(
Up vote post of Paul_Putter Down vote post of Paul_Putter
1.2k views
Posted by
Paul_Putter
Reply
Add a Comment

Replies

  • 1

Posted by
IRONMANROMEO
Up vote reply of IRONMANROMEO Down vote reply of IRONMANROMEO
Add a Comment
You can get the CVPixelBuffer from arView.session.currentFrame?.capturedImage, which comes without any Entities shown.

From there you can convert to whatever format you like, Google something like "CVPixelBuffer to UIImage/CGImage/etc."

Code Block swift
if let capImg = self.arView.session.currentFrame?.capturedImage {
    let ciimg = CIImage(cvPixelBuffer: capImg)
    let myImage = UIImage(ciImage: ciimg)
    // save image or whatever
}


Not accounting for rotation in my given example, there's examples of how to do that with a quick search.
Posted by
maxxfrazer
Post not yet marked as solved Up vote reply of maxxfrazer Down vote reply of maxxfrazer
Add a Comment
"I want to capture a photo of the underlying scene in RAW"

There is no way for you to capture a RAW photo of the underlying scene (In RealityKit, SceneKit, or ARKit) while an ARSession is running, I recommend that you file an enhancement request using Feedback Assistant to request that feature.
Posted by
gchiste
Post not yet marked as solved Up vote reply of gchiste Down vote reply of gchiste
Add a Comment