Hi All,
I apologive if this is a bit long. You can skip to part C to read the approach that I think could work (but has not worked for me yet).
We have m3u8 files hosted on a remote server. These files only point to video content; the master playlists do not include WebVTT content. The WebVTT file for each video is hosted remotely as well, but seperately.
We need a way to embed the subtitles into the video client-side in our iOS app (written in obj-c). After hacking away at this for a week, none of the following methods have worked (NOTE: each strategy was attempted seperately):
- This was the first medthod I tried (as described in the WWDC '12 session What's New In HTTP Live Streaming). At 19:00 - or, if you have the slides, the slide titled "Enabling (Non-Forced) Subtitles" - the presenter takes an asset, gets the mediaSelectionGroupForMediaChacteristics and adds it to an AVPlayerItem. However, the asset is derived from a master playlist file that already has the WebVTT embedded, so this is not helpful in my case.
- I created 2 AVURLAssets - one for my m3u8 file and one from my WebVTT file. I then created a AVMutableComposition, derived tracks from the 2 assets and inserted them into the composition. This did not work either. As described by an Apple staffer in this post: It is not possible to use AVMutableComposition to establish media selection groups for the subtitles, so you can't perform this sort of client side binding.
- Apple has sample code for a command line tool called avsubtitleswriter. When run from the command line, the tool takes (1) an input path with the location of the movie file, (2) an output path to write to and (3) an array of paths pointing to subtitle files. Ultimatlely, it "creates a new movie file at the specified output location, with audio, video, and subtitles from the input source, adding subtitles from the provide subtitles file(s). Each subtitles file will become a subtitle track in the output movie". Using the sample code as a template, I performed the following steps:
- Download the subtitle WebVTT data, turn it into a string, use NSRegularExpression to parse the string and create an array of custom subtitle objects (each one containing a time range and the subtitle text for that time range)...
- Create an AVAsset from the remote URL of the video whose master playlist does NOT contain the WebVTT data, read from it using AVAssetReader...
- 300+ lines of reading and writing that won't even execute because...
- YOU CAN'T USE AVASSETREADER ON AN ASSET WITH A REMOTE URL!!!!!!!!!!
- At this point, I arrive at a similar (!!unverified!!) conclusion to bdanis in this post: I have to "download the m3u8 file locally, edit it by adding the vtt url to it, and [give] that local file url to the AVAsset".
- In comes AVAssetResourceLoaderDelegate. Using Apple's AVURLDelegateDemo sample code and another gist I found (see [1] at the bottom of the page) as a template, I:
- Create an AVURLAsset using a URL that is nearly identical to the reomte URL, but with a custom scheme (non http or https)...
- Create a class that conforms to AVAssetResourceLoaderDelegate and assign it to the asset's resourceLoader. Start loading the requested keys asyncronously...
- In "shouldWaitForLoadingOfResponse" (an AVAssetResourceLoaderDelegate method) I attempt to catch any resource request from the asset.
- In the same method, I lazily inititalize an NSURLConnection. This connection is initialzed with the ORIGINAL HTTP ADDRESS OF THE REMOTE VIDEO. The same object that conforms to AVAssetResourceLoaderDelegate also conforms to NSURLConnectionDataDelegate, so I initalize the NSURLConnection with that object in the "delegate" paramater as well...
- I hook into NSURLConnectionDelegate methods. In "connection:didReceiveData:" I write to a NSMutableData object that I have stored as a property. In "connectionDidFinishLoading:" I write the data to a path in the user's document directory...
- Using the file path from #3.5, I try to read from that path in my AVAssetReader from #1.3.
- None of the AVAssetResourceLoaderDelegate methods are being called, so I decided to write this post.
There is so little documentation regarding adding WebVTT to an m3u8 file client-side. No matter how much research I do regarding a certain strategy, it's dificult to know if it will work until I (1) get some AVFoundation-generated error, (2) google that error and (3) delete all of the test cases & code that I just wrote because, alas, you cannot "use AVMutableComposition for X" or "use AVAssetReader to read from Y".
I can keep trying to refine my hybrid AVAssetReader/Writer + AVAssetResourceLoaderDeleagte apporach, but at this point I'd like to know if anyone - ANYONE - has tried something similar, has run into similar issues or has any sugestions. I haven't gotten the AVAssetResourceLoaderDelegate part (section C, #3) to work yet, and that is just so that I can have an AVURLAsset with a local URL so the other 300+ lines of the avsubtitleswriter-inspired reading & writing code (section C, #1) can run...god knows what runtime errors await me there.
I'd greatly appreciate some feedback. Thanks for taking the time to read this!
