How to render HDR content on iOS

Question

Created Oct ’20

Replies 6

Boosts 0

Views 3.7k

Participants 5

I notice that when I open the Photos app on my iPhone 12 Pro, viewing Photos or Videos shot in HDR makes them brighter than the overall display brightness level.

On macOS, there are APIs like EDRMetadata on CAMetalLayer and maximumExtendedDynamicRangeColorComponentValue on NSScreen.

I did see
CAMetalLayer.wantsExtendedDynamicRangeContent, but I'm not sure if this does what I'm looking for.

The Using Color Spaces to Display HDR Content documentation page describes setting the .colorspace on the CAMetalLayer for BT2020_PQ content, but it's not clear if this is referring to macOS or iOS. Is that the right way to get colors to be "brighter" than 1.0 on "XDR" mobile displays?

Boost

Answer 1

Graphics and Games Engineer OP

Apple

Oct ’20

Hi darknoon.

CAMetalLayer.wantsExtendedDynamicRangeContent is only available on macOS. Metal does not expose a way to display HDR content on iOS devices. HDR can only be done with videos and images.

By setting the CAMetalLayer.pixelFormat to on of the XR formats (like MTLPixelFormatBGR10_XR_sRGB), Metal does provide a way to get some extra headroom for better tone mapped HDR effects than what standard sRGB. Although this is not true HDR like HDR10 or Dolby Vision it does improve the effect.

0

Answer 2

oviano OP

Feb ’21

So is this situation going to change? Is the iOS and tvOS SDK going to be enhanced so that we can render HDR content properly using Metal layers as seems possible on macOS?

0

Answer 3

Graphics and Games Engineer OP

Apple

Feb ’21

oviano,

I would suggest creating a "Developer Tools" feedback ticket via Feedback Assistant and include you use case.

The more requests we get for any feature, the more likely you will see it in a future product and/or release.

Dan

0

Answer 4

Sol LeWitt OP

Feb ’21

I'm also facing this issue. My iOS app does HDR-aware image processing, and I'm able to see HDR/EDR values from my CIFilters when applied to HDR video within SwiftUI's VideoPlayer.

However, there doesn't seem to be any way to display HDR photos on the iPhone with EDR, the way they appear in Photos.app. It is probably possible to produce a one-frame HDR video using the input photo, but I'm trying to avoid this hack.

Filed as FB9008409

0

Answer 5

Graphics and Games Engineer OP

Apple

Feb ’21

Thanks. I'll make sure it gets to the CoreImage team.

1

Answer 6

darknoon OP

Apr ’21

Forgot to mention: I filed a more in depth description and sample project in FB8835138. For custom video HDR rendering (outside of AVPlayerLayer) I filed FB8834833

0