AVFoundation methods for smooth playback

Question

Devgeek OP

Created Oct ’20

Replies 1

Boosts 0

Participants 2

Dear Apple Engineers,

There seems to incomplete documentation about new AVFoundation methods introduced in iOS 14

Code Block  @available(iOS 13.0, *)   optional func anticipateRendering(using renderHint: AVVideoCompositionRenderHint) 
 @available(iOS 13.0, *) 
optional func prerollForRendering(using renderHint: AVVideoCompositionRenderHint)

How do we exactly use these methods, when are they exactly called, how much time can we take to load texture from memory in these methods, what is the correct usage...can anyone explain?

Boost

Answer 1

Media Engineer OP

Apple

Oct ’20

Here is the headerdoc from AVVideoCompositing.h:

/*!

@method anticipateRenderingUsingHint:

@abstract Informs a custom video compositor about upcoming rendering requests.

@param renderHint

Information about the upcoming composition requests.

@discussion

In the method the compositor can load composition resources such as overlay images which will be needed in the anticipated rendering time range.

Unlike -startVideoCompositionRequest, which is invoked only when the frame compositing is necessary, the framework typically calls this method every frame duration. It allows the custom compositor to load and unload a composition resource such as overlay images at an appropriate timing.

In forward playback, renderHint's startCompositionTime is less than endCompositionTime. In reverse playback, its endCompositionTime is less than startCompositionTime. For seeking, startCompositionTime == endCompositionTime, which means the upcoming composition request time range is unknown and the compositor shouldn’t preload time associated composition resources eagerly.

The method is guaranteed to be called before -startVideoCompositionRequest: for a given composition time.

The method is synchronous. The implementation should return quickly because otherwise the playback would stall and cause frame drops.

/
(void)anticipateRenderingUsingHint:(AVVideoCompositionRenderHint *)renderHint APIAVAILABLE(macos(10.15), ios(13.0), tvos(13.0)) SPIAVAILABLE(watchos(6.0));

/*!

@method prerollForRenderingUsingHint:

@abstract Tell a custom video compositor to perform any work in prerolling phase.

@param renderHint

Information about the upcoming composition requests.

@discussion

The framework may perform prerolling to load media data to prime the render pipelines for smoother playback. This method is called in the prerolling phase so that the compositor can load composition resources such as overlay images which will be needed as soon as the playback starts.

Not all rendering scenarios use prerolling. For example, the method won't be called while seeking.

If called, the method is guaranteed to be invoked before the first -startVideoCompositionRequest: call.

The method is synchronous. The prerolling won't finish until the method returns.

/
(void)prerollForRenderingUsingHint:(AVVideoCompositionRenderHint *)renderHint APIAVAILABLE(macos(10.15), ios(13.0), tvos(13.0)) SPIAVAILABLE(watchos(6.0));

If you are still unclear on how to use these methods, please file a report using the feedback assistant.

0