I have a use case to rotate a CMSampleBuffer from landscape to portrait. I have written a rough code for it. But still am facing many issues with appending of the sampleBuffer to input frame.
Here's the code:
`guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
return nil
}
// Get the dimensions of the image buffer
let width = CVPixelBufferGetWidth(imageBuffer)
let height = CVPixelBufferGetHeight(imageBuffer)
// Determine if the image needs to be rotated
let shouldRotate = width > height
// Create a CIImage from the buffer
var image = CIImage(cvImageBuffer: imageBuffer)
// Rotate the CIImage if necessary
if shouldRotate {
image = image.oriented(forExifOrientation: 6) // Rotate 90 degrees clockwise
}
let originalPixelFormatType = CVPixelBufferGetPixelFormatType(imageBuffer)
// Create a new pixel buffer
var newPixelBuffer: CVPixelBuffer?
let status = CVPixelBufferCreate(kCFAllocatorDefault, height, width,
originalPixelFormatType, nil, &newPixelBuffer)
guard status == kCVReturnSuccess, let pixelBuffer = newPixelBuffer else {
return nil
}
CVBufferPropagateAttachments(imageBuffer, newPixelBuffer!)
// Render the rotated image onto the new pixel buffer
let context = CIContext()
context.render(image, to: pixelBuffer)
CVPixelBufferUnlockBaseAddress(pixelBuffer,CVPixelBufferLockFlags(rawValue: 0))
var videoInfo: CMVideoFormatDescription?
CMVideoFormatDescriptionCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: newPixelBuffer!, formatDescriptionOut: &videoInfo)
var sampleTimingInfo = CMSampleTimingInfo(duration: CMSampleBufferGetDuration(sampleBuffer), presentationTimeStamp: CMSampleBufferGetPresentationTimeStamp(sampleBuffer), decodeTimeStamp: CMSampleBufferGetDecodeTimeStamp(sampleBuffer))
var newSampleBuffer: CMSampleBuffer?
CMSampleBufferCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: newPixelBuffer!, dataReady: true, makeDataReadyCallback: nil, refcon: nil, formatDescription: videoInfo!, sampleTiming: &sampleTimingInfo, sampleBufferOut: &newSampleBuffer)
let attachments: CFArray! = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: true)
let dictionary = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0),
to: CFMutableDictionary.self)
if let attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: true) as? [CFDictionary] {
for attachment in attachmentsArray {
for (key, value) in attachment as! Dictionary<CFString, Any> {
if let value = value as? CFTypeRef {
CMSetAttachment(newSampleBuffer!, key: key, value: value, attachmentMode: kCMAttachmentMode_ShouldPropagate)
}
}
}
}
return newSampleBuffer!
The error that I am getting while appending the frame is:
Error occured, isVideo = false, status = 3, Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12780), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x282b87390 {Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"}}
I read online that this error might be due to different PixelFormatType. But how can that be because I am obtaining the PixelFormatType from the buffer itself.
If you want to see the difference between original and rotated sample buffer. https://www.diffchecker.com/V0a55kCB/
Thanks in advance!