After updating to iOS 14.5.1 on the iPhone 12 Pro and iPhone 12 Pro Max at hand, I have observed a performance degradation in the process of acquiring camera images in ARKit.
The simplest code to reproduce this looks like this
class ViewController: UIViewController, ARSCNViewDelegate {
@IBOutlet var sceneView: ARSCNView!
// ...
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
guard let frame = sceneView.session.currentFrame else {return}
let pixelBuffer = frame.capturedImage
DispatchQueue.main.async(execute: {
let startTime = NSDate.timeIntervalSinceReferenceDate
let context = CIContext(options:nil)
let orientation = UIApplication.shared.statusBarOrientation
let viewportSize = self.sceneView.bounds.size
var image = CIImage(cvPixelBuffer: pixelBuffer)
let transform = frame.displayTransform(for: orientation, viewportSize: viewportSize).inverted()
image = image.transformed(by: transform)
guard let cameraImage = context.createCGImage(image, from: image.extent) else {return}
let endTime = NSDate.timeIntervalSinceReferenceDate
print(endTime - startTime)
})
}
// ...
}
The code for the entire project, which you can build and try out, is uploaded below.
https://github.com/thorikawa/arkit-snapshot-performance/archive/refs/heads/main.zip
This acquires the image from the current ARFrame as a CGImage and displays the elapsed time in the console. On an iPhone 12 Pro with iOS 14.4.2, it takes about 8-10 milliseconds, while on an iPhone 12 Pro with iOS 14.5.1, it takes about 18-20 milliseconds, sometimes over 30 milliseconds.
Is this a problem with the iOS and will performance return to normal in the future? Also, is there any way to avoid the performance degradation in the 14.5.1 version?