Rotate a CMSampleBuffer in swift5

Question

Created Nov ’23

Replies 0

Boosts 0

Participants 1

I have a use case to rotate a CMSampleBuffer from landscape to portrait. I have written a rough code for it. But still am facing many issues with appending of the sampleBuffer to input frame.

Here's the code:

`guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return nil }

// Get the dimensions of the image buffer
let width = CVPixelBufferGetWidth(imageBuffer)
let height = CVPixelBufferGetHeight(imageBuffer)

// Determine if the image needs to be rotated
let shouldRotate = width > height

// Create a CIImage from the buffer
var image = CIImage(cvImageBuffer: imageBuffer)

// Rotate the CIImage if necessary
if shouldRotate {
    image = image.oriented(forExifOrientation: 6)  // Rotate 90 degrees clockwise
}

let originalPixelFormatType = CVPixelBufferGetPixelFormatType(imageBuffer)


// Create a new pixel buffer
var newPixelBuffer: CVPixelBuffer?
let status = CVPixelBufferCreate(kCFAllocatorDefault, height, width,
                                 originalPixelFormatType, nil, &newPixelBuffer)
guard status == kCVReturnSuccess, let pixelBuffer = newPixelBuffer else {
    return nil
}

CVBufferPropagateAttachments(imageBuffer, newPixelBuffer!)


// Render the rotated image onto the new pixel buffer
let context = CIContext()
context.render(image, to: pixelBuffer)

CVPixelBufferUnlockBaseAddress(pixelBuffer,CVPixelBufferLockFlags(rawValue: 0))

        var videoInfo: CMVideoFormatDescription?

CMVideoFormatDescriptionCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: newPixelBuffer!, formatDescriptionOut: &videoInfo)

        var sampleTimingInfo = CMSampleTimingInfo(duration: CMSampleBufferGetDuration(sampleBuffer), presentationTimeStamp: CMSampleBufferGetPresentationTimeStamp(sampleBuffer), decodeTimeStamp: CMSampleBufferGetDecodeTimeStamp(sampleBuffer))

        var newSampleBuffer: CMSampleBuffer?

CMSampleBufferCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: newPixelBuffer!, dataReady: true, makeDataReadyCallback: nil, refcon: nil, formatDescription: videoInfo!, sampleTiming: &sampleTimingInfo, sampleBufferOut: &newSampleBuffer)

let attachments: CFArray! = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: true)
    let dictionary = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0),
        to: CFMutableDictionary.self)

if let attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: true) as? [CFDictionary] {
    for attachment in attachmentsArray {
        for (key, value) in attachment as! Dictionary<CFString, Any> {
            if let value = value as? CFTypeRef {
                CMSetAttachment(newSampleBuffer!, key: key, value: value, attachmentMode: kCMAttachmentMode_ShouldPropagate)
            }
        }
    }
}


return newSampleBuffer!

The error that I am getting while appending the frame is:

Error occured, isVideo = false, status = 3, Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12780), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x282b87390 {Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"}}

I read online that this error might be due to different PixelFormatType. But how can that be because I am obtaining the PixelFormatType from the buffer itself.

If you want to see the difference between original and rotated sample buffer. https://www.diffchecker.com/V0a55kCB/

Thanks in advance!

Boost