Processing ProRes video in Metal

I'm working on feeding a AVPlayer into a Metal view. The current test video I'm playing is ProRes 422.

If I read the buffer as RGB pixels (kCVPixelFormatType_32BGRA), then all works as expected. But this pixel format uses more processing on the ProRes decoder, and per Apple's WWDC session the preferred format is kCVPixelFormatType_422YpCbCr16.

So there's a couple issues I'm running into when using the YpCbCr format..

Issues / Questions:

  • Is there a set of fragment shader code to convert the texture from YpCbCr to RGB? When changing the pixel format, everything becomes grainy / distorted. I assume I need a function in the Metal pipeline to convert it. Is a fragment shader even the best way to handle converting it? I need it to get to RGB for rendering to a view.

  • Is there a major difference between kCVPixelFormatType_422YpCbCr16 and kCVPixelFormatType_422YpCbCr16BiPlanarVideoRange. I know the latter brings the texture in as two planes (which seems easier and more performant per activity monitor), but Apple recommended traditional kCVPixelFormatType_422YpCbCr16 with a single plane. Just curious what the best option to use is.

Processing ProRes video in Metal
 
 
Q