Recording ARKit as Video

Hi, Is there a performant way to record the ARKit view as a video?


- I've tried to use ReplayKit, but the methods are just not getting called.
- I've tried the ARSessionDelegate call backs to get a CVPixelBuffer, but it's not a fully rendered frame.


Has anyone got a solution?

One method is to use `RPScreenRecorder.shared().startCapture` instead of `RPScreenRecorder.shared().startRecording`

But this will capture the entire screen, not just the ARSCNView

RPScreenRecorder.shared().startRecording works for me but when I dismiss the previewViewController, the ARSession is frozen.

Which physical iPhone device are you using? I was tring on an iPhone SE

I had this problem too on 6s. Solved with recreation ARSCNView after dismissing preview controller.

But ReplayKit also captured all controls. Is there any way not to capture them?

The only *solution* Ive found is just when I record, show a count down, then hide all of the controls and record, when the user taps the screen, stop recording and bring back the controls. ugh.

I found a solution. You can use SCNRenderer with ARSCNView scene property. Take scene snapshots using `- (UIImage *)snapshotAtTime:withSize:antialiasingMode:` and combine them into video.

I just published a new framework for ARKit, called ARVideoKit.


ARVideoKit enables developers to capture and record ARKit videos, photos, Live Photos, and GIFs!


Using SCNRenderer I was able to render frames from ARSCNView and using material diffusing I was able to render ARSKView frames too!


Which means, NO MORE screen recording / screenshots to capture media of your awesome ARKit apps!!


Check out the framework here: https://github.com/AFathi/ARVideoKit

Hi Amhed,

That sounds pretty cool.


Does it work only with ARSCNViews or will any SCNView work?

How can I attach audio at the same time?


Thanks!

Okay that makes sense -- how do you combine them to a video?

And how to add audio?

Hey @drewster,


I just recently open-sourced ARVideoKit and that will allow you to easily configure the framework and use both AR and non-AR SCNView/SKView!


Feel free to clone/fork the project: https://www.github.com/AFathi/ARVideoKit


Additionally, ARVideoKit allows you to easily record audio while rendering a scene's content.


-Ahmed

Thanks Ahmed,


Your pod seems ideal, but it makes my app freeze!

AR is already quite expensive and it seems that your code adds another renderer on top?


console log:

Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4)

Hey,


I solved this by creating a HUD that is on a different window. ReplayKit doesn't record this.


https://medium.com/ar-tips-and-tricks/how-to-record-a-screen-capture-with-replaykit-whilst-hiding-the-hud-element-bedcca8e31e

Hey Ahmed,


I tried using this and it would record my objects in the wrong location - do you know why this may happen ?


O

I achieved this the same way you would record a video from a camera really - AVAssetWriter. The only difference being your captured frame CVPixelBuffers don't come from an AVCaptureSession, they come from your ARFrames which are provided to you in the session(_:,didUpdate:) callback.


ARCapture framework allows to record video from ARKit. It's lightweight, well documented and can be extended if needed. Moreover, it has no memory leaks.

Recording ARKit as Video
 
 
Q