Render Metal Video into AVPlayerLayer?

We use a custom video player that gets frames out of an RTSP stream and renders them into a Metal view. It works well.

I'm trying to parse out how to translate this to support Picture in Picture. AVPictureInPictureController takes a AVPlayerLayer which takes a AVPlayer which is formed either from a URL or media list. Neither of which are applicable here.

Can I render the metal video into the AVPlayerLayer directly? That seems like the right path, but some of the documentation implies our app won't be called to render frames even if the PiP background mode is turned on in that case.

Basically, I'm trying to get a clear answer to "can I render video with PiP using Metal". As there is no "URL" I can pass for this, that is the only path that seems available. Thanks.
Up vote post of cypher Down vote post of cypher
1.5k views
Posted by
cypher
Reply
Add a Comment

Replies

AVSampleBufferDisplayLayer can be used to render video frames that you produce yourself, but AVSBDL does not support PIP. So the answer is no, there is no current way to render custom video frames using AVFoundation in a way that supports PIP. We welcome your feature requests, if you can describe your use case in a feedback assistant support.
Posted by
Media Engineer
Up vote reply of Media Engineer Down vote reply of Media Engineer

  • I have a use case to render graphics in a SampleBuffer via OpenGL during video call, but for our case when users go in a PiP mode, all graphics does not able to render. Is there any way to allow GPU in PiP mode ?

    —
    amadani
Add a Comment