captureHighResolutionFrame

Converting YCbCr to RGB with Metal is only needed if you intend to render the captured image with Metal (sample project demonstrating how). The simplest way to obtain a PNG image is with CoreImage:

// The file can be accessed in Finder if you set UIFileSharingEnabled to YES in your
// Info.plist.
guard let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first else { return }

// If you want to save multiple PNG images, it's best practice to reuse the same CIContext.
let context = CIContext()

let image = CIImage(cvPixelBuffer: frame.capturedImage)
let png = context.pngRepresentation(of: image, format: .BGRA8, colorSpace: image.colorSpace!)
try? png?.write(to: documentsURL.appending(component: "captured-image.png"))

Hello,

I recommend leveraging CoreImage for this, for example:

        session.captureHighResolutionFrame { frame, error in
            guard let frame else { return }
            
            let ciImage = CIImage(cvPixelBuffer: frame.capturedImage)
                    
            do {
                try ciContext.writePNGRepresentation(of: ciImage, to: outputURL, format: .RGBA8, colorSpace: ciImage.colorSpace!)
            } catch {
                fatalError(error.localizedDescription)
            }
        }

Thank you for your reply.

In fact I noticed a color problem that occurs after correcting the orientation of the image (the captured image is in landscape format)...

The corrected image has slightly different colors from the uncorrected version.

Here is the code I use to correct the orientation:

let pixelBufferRef = frame.capturedImage

let resolution = frame.camera.imageResolution

var image = CIImage(cvPixelBuffer: pixelBufferRef)

let viewportSize = CGSize(width: resolution.height, height: resolution.width)

let transform = frame.displayTransform(for: .portraitUpsideDown, viewportSize: viewportSize)

image = image.transformed(by: transform)

let context = CIContext()

if let imageRef = context.createCGImage(image, from: image.extent) {
    let png = context.pngRepresentation(of: image, format: .BGRA8, colorSpace: image.colorSpace!)
   
    try? png?.write(to: documentsURL.appending(component: "captured-image-corrected.png"))
}

Did I miss something?

if let imageRef = context.createCGImage(image, from: image.extent) {
    let png = context.pngRepresentation(of: image, format: .BGRA8, colorSpace: image.colorSpace!)
   
    try? png?.write(to: documentsURL.appending(component: "captured-image-corrected.png"))
}

I'm not sure what the purpose of the createCGImage call is here? Can you log the CIImage's colorSpace before and after, also log it before and after the transform, to make sure it hasn't changed, and then post the results to this thread?

Thank you!

The color spaces are identical.

Before:

<CGColorSpace 0x281da9440> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; QuickTime 'nclc' Video (1,1,6))

After:

<CGColorSpace 0x281da9440> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; QuickTime 'nclc' Video (1,1,6))

But you are absolutely right, the code I posted was wrong!

Here is a cleaner version that seems to work

let pixelBufferRef = frame.capturedImage

let resolution = frame.camera.imageResolution

var image = CIImage(cvPixelBuffer: pixelBufferRef)

let viewportSize = CGSize(width: resolution.height, height: resolution.width)

let transform = frame.displayTransform(for: .portraitUpsideDown, viewportSize: viewportSize)

image = image.transformed(by: transform)

do {
    let context = CIContext()
    
    let url = getDocumentsDirectory().appending(component: "captured-image-corrected.png")

    try context.writePNGRepresentation(of: image, to: url, format: .RGBA8, colorSpace: image.colorSpace!)
} catch {
    fatalError(error.localizedDescription)
}

Does this sound better to you?