After updating to iOS 14.5.1 on the iPhone 12 Pro and iPhone 12 Pro Max at hand, I have observed a performance degradation in the process of acquiring camera images in ARKit.
The simplest code to reproduce this looks like this
The code for the entire project, which you can build and try out, is uploaded below.
https://github.com/thorikawa/arkit-snapshot-performance/archive/refs/heads/main.zip
This acquires the image from the current ARFrame as a CGImage and displays the elapsed time in the console. On an iPhone 12 Pro with iOS 14.4.2, it takes about 8-10 milliseconds, while on an iPhone 12 Pro with iOS 14.5.1, it takes about 18-20 milliseconds, sometimes over 30 milliseconds.
Is this a problem with the iOS and will performance return to normal in the future? Also, is there any way to avoid the performance degradation in the 14.5.1 version?
The simplest code to reproduce this looks like this
Code Block class ViewController: UIViewController, ARSCNViewDelegate { @IBOutlet var sceneView: ARSCNView! // ... func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) { guard let frame = sceneView.session.currentFrame else {return} let pixelBuffer = frame.capturedImage DispatchQueue.main.async(execute: { let startTime = NSDate.timeIntervalSinceReferenceDate let context = CIContext(options:nil) let orientation = UIApplication.shared.statusBarOrientation let viewportSize = self.sceneView.bounds.size var image = CIImage(cvPixelBuffer: pixelBuffer) let transform = frame.displayTransform(for: orientation, viewportSize: viewportSize).inverted() image = image.transformed(by: transform) guard let cameraImage = context.createCGImage(image, from: image.extent) else {return} let endTime = NSDate.timeIntervalSinceReferenceDate print(endTime - startTime) }) } // ... }
The code for the entire project, which you can build and try out, is uploaded below.
https://github.com/thorikawa/arkit-snapshot-performance/archive/refs/heads/main.zip
This acquires the image from the current ARFrame as a CGImage and displays the elapsed time in the console. On an iPhone 12 Pro with iOS 14.4.2, it takes about 8-10 milliseconds, while on an iPhone 12 Pro with iOS 14.5.1, it takes about 18-20 milliseconds, sometimes over 30 milliseconds.
Is this a problem with the iOS and will performance return to normal in the future? Also, is there any way to avoid the performance degradation in the 14.5.1 version?