hdr dolby vision video color correction to 8 bit still images

Question

diffent OP

Created Mar ’22

Replies 2

Boosts 0

Views 1.9k

Participants 2

Hey guys, tried to follow the super confusing doc on this, but no luck yet.

https://developer.apple.com/av-foundation/Incorporating-HDR-video-with-Dolby-Vision-into-your-apps.pdf

I have code that uses AVAssetReader and AVAssetReaderTrackOutput to directly pull frames from a video, but the colors are wrong for HDR dolby videos. Basically what I want is to extract frames from an HDR Dolby video as images and have those images not be the wrong color. Don't care if they are only 8 bit per color instead of 10 and all the all new stuff, just the closest that old fashioned 8 bit per color supports.

I added the statement marked // added for dolby hdr per the above doc, (spread across several lines), no luck, still bad colors.

Any hints of what I am missing?

NSMutableDictionary* dictionary = [[NSMutableDictionary alloc] init];

[dictionary setObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(NSString*)kCVPixelBufferPixelFormatTypeKey];

// added for dolby hdr

dictionary[AVVideoColorPropertiesKey] = @{

AVVideoColorPrimariesKey: AVVideoColorPrimaries_ITU_R_709_2,

AVVideoTransferFunctionKey: AVVideoTransferFunction_ITU_R_709_2,

AVVideoYCbCrMatrixKey: AVVideoYCbCrMatrix_ITU_R_709_2

};

AVAssetReaderTrackOutput* asset_reader_output = [[AVAssetReaderTrackOutput alloc] initWithTrack:video_track outputSettings:dictionary];

[asset_reader addOutput:asset_reader_output];

// from here we get sample buffers like this

CMSampleBufferRef buffer2 = [asset_reader_output copyNextSampleBuffer];

// then pixel buffer

CVPixelBufferRef inputPixelBuffer = CMSampleBufferGetImageBuffer(buffer2);

// then a CIImage

CIImage* ciImage = [CIImage imageWithCVPixelBuffer:inputPixelBuffer]; // one vid frame

then we use standard stuff to convert that to a CGImage/UIImage

Boost

Answer 1

diffent OP

Oct ’22

fyi we had an apple office hours meeting to address this, this is the first part of the solution we found based on those discussions

pixel format needed to change, and 3 dictionary key values added to AVAssetReaderTrackOutput outputSettings: argument per below

with these, the UIImage seem to come out much better but we havent done a full check yet

the next step is to get these images written back to a video file w/o losing their color again

this is just a code snippet, if anyone needs to full code we can post the full xcode sample as we have extracted the buggy dolby stuff into a small test case

the info about the color properties (noted below) we had found previously in the apple docs about this technology, but it also needed the updated pixel key to cause any positive effect on the outputted images. that part was not clear from the docs immediately.

NSArray* video_tracks = [asset tracksWithMediaType:AVMediaTypeVideo]; 
AVAssetTrack* video_track = [video_tracks firstObject];
NSMutableDictionary* dictionary = [[NSMutableDictionary alloc] init];
// kCVPixelFormatType_32BGRA // this was the what we had
// 420YpCbCr8BiPlanarFullRange // this is the correct one (see next line)

[dictionary setObject:[NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarFullRange] forKey:(NSString*)kCVPixelBufferPixelFormatTypeKey];

// this is also needed

dictionary[AVVideoColorPropertiesKey] =

            @{AVVideoColorPrimariesKey:

                AVVideoColorPrimaries_ITU_R_709_2,

            AVVideoTransferFunctionKey:

                AVVideoTransferFunction_ITU_R_709_2,

            AVVideoYCbCrMatrixKey:

                AVVideoYCbCrMatrix_ITU_R_709_2};

        AVAssetReaderTrackOutput* asset_reader_output = [[AVAssetReaderTrackOutput alloc] initWithTrack:video_track outputSettings:dictionary];

0

Answer 2

lalitpatil1986 OP

Mar ’23

Can you please post sample code which you mentioned that would be great help. thanks in advance

0