How do I convert a CVPixelBuffer to Data?

Question

kart OP

Created Dec ’17

Replies 3

Boosts 0

Views 7.7k

Participants 3

My camera app captures a photo, enhances it in a certain way, and saves it.

To do so, I get the input image from the camera in the form of a CVPixelBuffer (wrapped in a CMSampleBuffer). I perform some modifications on the pixel buffer, and I then want to convert it to a Data object. How do I do this?

Note that I don't want to convert the pixel buffer / image buffer to a UIImage or CGImage since those don't have metadata (like EXIF). I need a Data object. How do I get one from a CVPixelBuffer / CVImageBuffer?

PS: I tried calling

AVCapturePhotoOutput.jpegPhotoDataRepresentation()

but that fails saying "Not a JPEG sample buffer". Which makes sense since the CMSampleBuffer contains a pixel buffer (a bitmap), not a JPEG.

Boost

Answer 1

QuinceyMorris OP

Dec ’17

You need to "lock" the data using "CVPixelBufferLockBaseAddress(_:_:)", then access the data using "CVPixelBufferGetBaseAddress(_:)". Then you can create a Data object using the bytes from this base address. Make sure you do not use one of the Data "bytesNoCopy" initializers to copy the data, because the buffer pointed to by the base address may not exist persistently.

Note that you may have to use "CVPixelBufferGetBaseAddressOfPlane(_:_:)" for each plane, if the pixel buffer is planar, and decide how to combine the planes into a single Data object.

Also, don't forget to use "CVPixelBufferUnlockBaseAddress(_:_:)" to unlock the pixel data after you've created the Data object.

0

Answer 2

kart OP

Dec ’17

Thanks, but I need a Data object containing a JPEG, to save to the Photos app. Sorry that I forgot to mention this in my original post. I don't want a Data object containing uncompressed data.

0

Answer 3

Planet30 OP

Feb ’21

... I know this is an old thread, but it looks like what I am trying to figure out. Maybe you could help me out ...

I have raw pixel data from a binary file and try to convert them into a CIImage. I decided to go via CVPixelBuffer in order to be able to provide the pixel layout (CFA) of my image sensor, which is a Sony RGGB (aka. "rgg4"). The rawImgData are [UInt16] with width*height number of elements, because I had to do some numerical treatment on the data.

This is how far I got so far. I am stuck with the pixel-buffer properties.

What do I have to put in the dictionary for CIFilter to create a raw image? I always get an error messages at line 10. The compiler does not like the "nil".

Code Block var pixelBuffer: CVPixelBuffer?
let attrs: [ CFString : Any ]  = [	kCVPixelBufferCGImageCompatibilityKey : kCFBooleanTrue as Any,
																		kCVPixelBufferCGBitmapContextCompatibilityKey : kCFBooleanTrue as Any ]
CVPixelBufferCreateWithBytes(kCFAllocatorDefault, width, height, kCVPixelFormatType_14Bayer_RGGB, &imgRawData, 2*width, nil, nil, attrs as CFDictionary, &pixelBuffer)
let sourceTypeIdentifierHint = CIRAWFilterOption(rawValue: kCGImageSourceTypeIdentifierHint as String)
let rfo: [CIRAWFilterOption: Any] = [ sourceTypeIdentifierHint : "com.sony.raw-image" ]
let ciiraw = CIFilter(cvPixelBuffer: pixelBuffer, properties: nil, options: rfo).outputImage

I put this in this thread, because one could imagine to convert the pixel buffer into data object and then omit the properties in CIFilter. But maybe, I would loose too much info from the pixel buffer. No clue.

Thank you for your help!

0