avassetreader convert hdr frames to proper colorized CIImages

Question

Created Jul ’21

Replies 3

Boosts 0

Views 1.3k

Participants 2

there are a lot of examples on how to add HDR video support to your apps, but we want to do is read video frames from an HDR video file and then properly map the colors to CIImages so they don't look washed out. it seems like there is some sort of color transform that needs to be applied. but does anyone know the magic formula?

CVPixelBufferRef inputPixelBuffer = CMSampleBufferGetImageBuffer(buffer);

CIImage* ciImage = [CIImage imageWithCVPixelBuffer:inputPixelBuffer]; // one vid frame

Boost

Answer 1

kargoedev OP

Jul ’21

seems like we are looking for some sort of HDR to SDR tonemapping CIFilter or perhaps a CGColorSpace that does this mapping

0

Answer 2

Media Engineer OP

Apple

Jul ’21

You might want to add CoreImage or CoreGraphics tags to the question

0

Answer 3

kargoedev OP

Oct ’22

FYI we got a half hour "office hours" appointment and after some chatter and trial and error, this method seemed to do the trick at extracting images from HDR / Dolby vids with better color. We don't know if it is the full answer yet, but it is definitely better.

        // prior to here, set up the AVAsset * asset 

       // code is a bit of archaic ObjC syntax but it works

        NSArray* video_tracks = [asset tracksWithMediaType:AVMediaTypeVideo];

        AVAssetTrack* video_track = [video_tracks firstObject];

        NSMutableDictionary* dictionary = [[NSMutableDictionary alloc] init];

        // kCVPixelFormatType_32BGRA (pixel format we originally used)
        // 420YpCbCr8BiPlanarFullRange (new pixel format that seems to work)

        // need to change the pixel format to this new one here, and also set the video color prop key below (which is 3 items in a dictionary)

        [dictionary setObject:[NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarFullRange

        ] forKey:(NSString*)kCVPixelBufferPixelFormatTypeKey];

        dictionary[AVVideoColorPropertiesKey] =

            @{AVVideoColorPrimariesKey:
                AVVideoColorPrimaries_ITU_R_709_2,
            AVVideoTransferFunctionKey:
                AVVideoTransferFunction_ITU_R_709_2,
            AVVideoYCbCrMatrixKey:
                AVVideoYCbCrMatrix_ITU_R_709_2};

        AVAssetReaderTrackOutput* asset_reader_output = [[AVAssetReaderTrackOutput alloc] initWithTrack:video_track outputSettings:dictionary];

// then proceed to normally walk thru frames of the video

0