Using WebVTT captions from sidecar

I also would love to know how to do this. I've pieced information together from several sources - basically I've gotten as far as needing to generate segmented .vtt files based off the larger single sidecar .vtt file. You can use mediafilesegmenter to generate these, but it would be great if this could be done on a server somewhere on the fly as needed (so we didn't need to generate 50k m3u8 files for all of our video assets).

It appears that all you need to add to the master m3u8 is this line:

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="eng",URI="http://myserver.com/mycaptions.m3u8"

but it seems that it expects the URI to be another manifest file listing all the segmented .vtt files. Any direction or further documentation on this would be very helpful.

vtcajones,

That is correct that you will need to include a line in the master playlist pointing to the playlist that lists the vtt segments for each subtitle track you would like to make available. You can either do this server-side, or you could download and modify the master m3u8 on the client, add the appropriate line(s), and then initialize the AVURLAsset using the modified master playlist as a file URL.

It would be best to include the relevant lines in the playlist residing on the server, but this should work as a fallback.

In addition to the suggestion by Tidbits, you could also use AVAssetResourceLoader and AVAssetResourceLoaderDelegate to implmenent shouldWaitForLoadingOfRequestedResource. This delegate method gives you the opertunity to handle URLs that the AVPlayer doesn't know how to process and provide it with the result. What you would want to do is fetch the manifest yourself, and rewrite it on the client to include a variant playlist that points at the WebVTT file. In order to get the AVAsset to call shouldWaitForLoadingOfRequestedResource, you would need to change the manifest URL to something like "PRIVATEPREFIXhttp://...".

One thing to keep in mind in either case, is that you will need to do additional rewriting in the manifest if the URLs inside are relative and not absolute.

You don't have to segment the VTT files.

If you are willing to require version 6 you can use EXT-X-MAP to describe the head of the VTT and then byte range into the rest.

Unfortunately, I don't believe mediasubtitlesegmenter is able to do this for you.

Here's an example of what the start of such a playlist might look like.

#EXTM3U
#EXT-X-TARGETDURATION:65
#EXT-X-VERSION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MAP:URI="foo.webvtt",BYTERANGE="57@0"
#EXTINF:63.36300,  
#EXT-X-BYTERANGE:731@57
foo.webvtt
#EXTINF:60.62800,
#EXT-X-BYTERANGE:1184@788
foo.webvtt
#EXTINF:62.19500,  
#EXT-X-BYTERANGE:1607@1972
foo.webvtt

Hi TidBits

We have simialr issue with our APPs. Just want to understand more about the two different methods to address this issue.

Server-side: In mater playlist, pointing to the playlist that lists the vtt segments for each subtitle track. Is that what we call inline subtitle.

Client-Side: The user App modify the m3u8 manifest to add the line(s). In such way, it changed from sidecar to inline. Is that correct?

Thanks in advance. FYI, I am not a developer but an administrator or supporter for our APP platform.

Alex

Hi Alex,

That is correct - those are the two available options. You could also use AVAssetResourceLoaderDelegate to modify the master playlist on the fly, which would actually be the preferred option for adjusting the playlist on the client side.

Here is a link to sample code that demonstrates the use of AVAssetResourceLoader:

https://developer.apple.com/library/content/samplecode/sc1791/Introduction/Intro.html

You're saying "sidecar" but I think you're describing what I would call "sideload". If that's true, I've succeeded at doing this, and I've dropped an explanation and my source code on StackOverflow: https://stackoverflow.com/a/67308098/15788389 and on github: https://github.com/kanderson-wellbeats/sideloadWebVttToAVPlayer. I apologize for linking out instead of posting stuff here directly but Apple does not let me post my full answer because it's too long.