How do I display WebVTT Closed Captions using AVPlayer?

Hello,


We have the following stream setup, with WebVTT closed captions in the master index. The question is:


(1) Can AVPlayer render WebVTT Closed Captions?

(2) How can I determine if WebVTT Closed Captions exist on an asset?

(3) How can I enable/disable WebVTT Closed Captions on the AVPlayer during playback?


Thanks,

Mike


index.m3u8
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="group00",NAME="lang",DEFAULT=YES,FORCED=NO,LANGUAGE="en",URI="04.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=353933,AVERAGE-BANDWIDTH=192512,RESOLUTION=320x180,CODECS="avc1.4d400c,mp4a.40.5",SUBTITLES="group00"
01.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=486658,AVERAGE-BANDWIDTH=264704,RESOLUTION=320x180,CODECS="avc1.4d400c,mp4a.40.5",SUBTITLES="group00"
02.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=694041,AVERAGE-BANDWIDTH=377504,RESOLUTION=384x216,CODECS="avc1.4d400d,mp4a.40.5",SUBTITLES="group00"
03.m3u8



04.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:7
#EXT-X-MEDIA-SEQUENCE:109580
#EXTINF:6,
20151102T165342-04-109580.vtt
#EXTINF:6,
20151102T165342-04-109581.vtt
#EXTINF:6,
20151102T165342-04-109582.vtt
#EXTINF:6,
20151102T165342-04-109583.vtt
#EXTINF:6,
20151102T165342-04-109584.vtt
#EXTINF:6,
20151102T165342-04-109585.vtt


20151102T165342-04-109581.vtt
WEBVTT
X-TIMESTAMP-MAP=MPEGTS:450000, LOCAL:291:38:00.895


302:17:48.200 --> 302:17:48.280 align:start position:71% line:91% size:56%
PLEASE!


302:17:48.280 --> 302:17:49.520 align:start position:65% line:91% size:68%
NO MORE FRIES!


302:17:49.520 --> 302:17:52.360 align:start position:69% line:91% size:60%
( GAGS )


302:17:52.360 --> 302:17:54.200 align:start position:64% line:91% size:70%
Steven: FRYBO, STOP!

bump - Does anyone know if AVPlayer can render WebVTT Closed Captions?

Yes , see apple's docs on that https://developer.apple.com/streaming/examples/ .

The advanced example has webvtt subtitles . Also AVPlayer has a property called

closedCaptionDisplayEnabled

I understand that the documentation says webvtt subtitles are supported. however, I still do not have proof that it works. I have tried enabling closedCaptionDisplayEnabled, and that has not helped either.


I have built a sample application that only uses the AVPlayer, and ~100 lines of code that represents my issue: https://ooyala.box.com/apple-86304


Steps to reproduce:

1. Load the following video in Quicktime Desktop: http://player.ooyala.com/player/all/92cWp0ZDpDm4Q8rzHfVK6q9m6OtFP-ww.m3u8. If you look at this video, it has valid webVTT subtitles in the master manifest.

2. Play the video, confirm that there is a "Closed captions" button that appears, and that you can choose from one of many languages. This confirms that Quicktime Desktop can recognize the webVTT files and display the closed captions

3. Download the application provided here: https://ooyala.box.com/apple-86304

4. Open the application. If you look in the ViewController.m, you can see the same url is loaded into an AVPlayer, with some observers. Note that I also set closedCaptionDisplayEnabled to YES

5. Play the app. Note a number of things:

5a. Closed captions will not appear

5b. In the xcode logs, I print out the tracks available for the asset, and there is only "vide" and "soun", no "sbtl" tracks, like those found in the BipBop asset (You can try the BipBop by uncommenting it on line 23)


There is no evidence that webVTT subtitles are supported, or that there is a way to do this with the AVPlayer. Can you please advise?


Thanks,

Michael

I stumbled upon this post in my search for the same answers as you.

Turns out closedCaptionDisplayEnabled does nothing in this scenario.

I found the solution in this answer. The key to getting this to work is this method selectMediaOption(_:inMediaSelectionGroup:)


Here’s how I got this working:

if let mediaSelectionGroup = currentItem?.asset.mediaSelectionGroupForMediaCharacteristic(AVMediaCharacteristicLegible) {
  currentItem?.selectMediaOption(mediaSelectionGroup.options[0], inMediaSelectionGroup: mediaSelectionGroup)
}


This is of course just selecting the first available subtitle option. You will have to display a list of options and then select the correct option depending on the user’s selection. Hope this helps.

Have a look at this: http://asciiwwdc.com/2013/sessions/608


It mentions the sample code "AVLegibleMeanings" and "avsubtitleswriter for OS X"

Are you using AVPlayerLayer or AVPlayerViewController?


If the latter, the video player will provide a subtitle selection UI. If AVPlayerLayer you need to provide your own UI for selecting subtitles. In either case, the correct subtitles should be automatically selected and displayed based on accessibility preferences set on the device.


When managing the display yourself, selecting the correct media option is (as described in the above post) is the correct way to programmatically select which subtitle track to display.

We’ve implemented our own UI using AVPlayerLayer and therefore needed to manually control the the subtitle selection.

However, it seems if I select the subtitle too early after the video starts, the subtitle won’t get activated. If I deselect (pass nil into selectMediaOption(_:inMediaSelectionGroup:)) and select it again, it gets activated. Very weird.

Strike that. It seems to be a Simulator issue.

I got this working. It was a horrible experience but I did succeed. My full explanation is too long to post here so I've stuck my long explanation and sample code on Github: https://github.com/kanderson-wellbeats/sideloadWebVttToAVPlayer . I dearly hope my link doesn't die. Apple doesn't want us posting long answers to questions but they're just fine forcing us to do weeks of ridiculous work for something that on another platform is an hour or less.
I have been looking at kanderson's code and trying to understand it. Considering the fact that it was posted a week ago the code is written in ancient Swift that I am unable to translate into modern Swift that will compile. If there was an example that is recent that would be appreciated.
How do I display WebVTT Closed Captions using AVPlayer?
 
 
Q