Problem with Advanced stream (HEVC/H.264) example

Hi

There's a multi codec h264/hevc hls example on https://developer.apple.com/streaming/examples/

I am wondering if media playlist in this example is ok, it uses two codecs for rendering video but doesn't

specify rendition groups for video and I'm wondering how client should figure out not to mix fragments with

different codec.

Thanks.

Łukasz

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS


#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a1",NAME="English",LANGUAGE="en-US",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",URI="a1/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a2",NAME="English",LANGUAGE="en-US",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="6",URI="a2/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a3",NAME="English",LANGUAGE="en-US",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="6",URI="a3/prog_index.m3u8"


#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS,GROUP-ID="cc",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,INSTREAM-ID="CC1"


#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="sub1",LANGUAGE="en",NAME="English",AUTOSELECT=YES,DEFAULT=YES,FORCED=NO,URI="s1/en/prog_index.m3u8"


#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=928091,BANDWIDTH=1015727,CODECS="avc1.640028",RESOLUTION=1920x1080,URI="tp5/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=731514,BANDWIDTH=760174,CODECS="avc1.64001f",RESOLUTION=1280x720,URI="tp4/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=509153,BANDWIDTH=520162,CODECS="avc1.64001f",RESOLUTION=960x540,URI="tp3/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=176942,BANDWIDTH=186651,CODECS="avc1.64001f",RESOLUTION=640x360,URI="tp2/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=90796,BANDWIDTH=95410,CODECS="avc1.64001f",RESOLUTION=480x270,URI="tp1/iframe_index.m3u8"


#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2190673,BANDWIDTH=2523597,CODECS="avc1.640020,mp4a.40.2",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v5/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=8052613,BANDWIDTH=9873268,CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v9/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=6133114,BANDWIDTH=7318337,CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v8/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=4681537,BANDWIDTH=5421720,CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v7/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=3183969,BANDWIDTH=3611257,CODECS="avc1.640020,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v6/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1277747,BANDWIDTH=1475903,CODECS="avc1.64001f,mp4a.40.2",RESOLUTION=768x432,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v4/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=890848,BANDWIDTH=1017705,CODECS="avc1.64001f,mp4a.40.2",RESOLUTION=640x360,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v3/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=533420,BANDWIDTH=582820,CODECS="avc1.64001f,mp4a.40.2",RESOLUTION=480x270,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v2/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=303898,BANDWIDTH=339404,CODECS="avc1.64001f,mp4a.40.2",RESOLUTION=416x234,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v1/prog_index.m3u8


#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2413172,BANDWIDTH=2746096,CODECS="avc1.640020,ac-3",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v5/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=8275112,BANDWIDTH=10095767,CODECS="avc1.64002a,ac-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v9/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=6355613,BANDWIDTH=7540836,CODECS="avc1.64002a,ac-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v8/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=4904036,BANDWIDTH=5644219,CODECS="avc1.64002a,ac-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v7/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=3406468,BANDWIDTH=3833756,CODECS="avc1.640020,ac-3",RESOLUTION=1280x720,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v6/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1500246,BANDWIDTH=1698402,CODECS="avc1.64001f,ac-3",RESOLUTION=768x432,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v4/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1113347,BANDWIDTH=1240204,CODECS="avc1.64001f,ac-3",RESOLUTION=640x360,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v3/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=755919,BANDWIDTH=805319,CODECS="avc1.64001f,ac-3",RESOLUTION=480x270,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v2/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=526397,BANDWIDTH=561903,CODECS="avc1.64001f,ac-3",RESOLUTION=416x234,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v1/prog_index.m3u8


#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2221172,BANDWIDTH=2554096,CODECS="avc1.640020,ec-3",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v5/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=8083112,BANDWIDTH=9903767,CODECS="avc1.64002a,ec-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v9/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=6163613,BANDWIDTH=7348836,CODECS="avc1.64002a,ec-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v8/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=4712036,BANDWIDTH=5452219,CODECS="avc1.64002a,ec-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v7/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=3214468,BANDWIDTH=3641756,CODECS="avc1.640020,ec-3",RESOLUTION=1280x720,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v6/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1308246,BANDWIDTH=1506402,CODECS="avc1.64001f,ec-3",RESOLUTION=768x432,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v4/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=921347,BANDWIDTH=1048204,CODECS="avc1.64001f,ec-3",RESOLUTION=640x360,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v3/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=563919,BANDWIDTH=613319,CODECS="avc1.64001f,ec-3",RESOLUTION=480x270,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v2/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=334397,BANDWIDTH=369903,CODECS="avc1.64001f,ec-3",RESOLUTION=416x234,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v1/prog_index.m3u8


#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=287207,BANDWIDTH=328352,CODECS="hvc1.2.4.L123.B0",RESOLUTION=1920x1080,URI="tp10/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=216605,BANDWIDTH=226274,CODECS="hvc1.2.4.L123.B0",RESOLUTION=1280x720,URI="tp9/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=154000,BANDWIDTH=159037,CODECS="hvc1.2.4.L123.B0",RESOLUTION=960x540,URI="tp8/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=90882,BANDWIDTH=92800,CODECS="hvc1.2.4.L123.B0",RESOLUTION=640x360,URI="tp7/iframe_index.m3u8"
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=50569,BANDWIDTH=51760,CODECS="hvc1.2.4.L123.B0",RESOLUTION=480x270,URI="tp6/iframe_index.m3u8"


#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1966314,BANDWIDTH=2164328,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v14/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=6105163,BANDWIDTH=6664228,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v18/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=4801073,BANDWIDTH=5427899,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v17/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=3441312,BANDWIDTH=4079770,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v16/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2635933,BANDWIDTH=2764701,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v15/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1138612,BANDWIDTH=1226255,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=768x432,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v13/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=829339,BANDWIDTH=901770,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=640x360,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v12/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=522229,BANDWIDTH=548927,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=480x270,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v11/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=314941,BANDWIDTH=340713,CODECS="hvc1.2.4.L123.B0,mp4a.40.2",RESOLUTION=416x234,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a1",SUBTITLES="sub1"
v10/prog_index.m3u8


#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2188813,BANDWIDTH=2386827,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v14/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=6327662,BANDWIDTH=6886727,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v18/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=5023572,BANDWIDTH=5650398,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v17/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=3663811,BANDWIDTH=4302269,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v16/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2858432,BANDWIDTH=2987200,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=1280x720,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v15/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1361111,BANDWIDTH=1448754,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=768x432,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v13/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1051838,BANDWIDTH=1124269,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=640x360,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v12/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=744728,BANDWIDTH=771426,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=480x270,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v11/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=537440,BANDWIDTH=563212,CODECS="hvc1.2.4.L123.B0,ac-3",RESOLUTION=416x234,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a2",SUBTITLES="sub1"
v10/prog_index.m3u8


#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1996813,BANDWIDTH=2194827,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=960x540,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v14/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=6135662,BANDWIDTH=6694727,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v18/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=4831572,BANDWIDTH=5458398,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v17/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=3471811,BANDWIDTH=4110269,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=1920x1080,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v16/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2666432,BANDWIDTH=2795200,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=1280x720,FRAME-RATE=60.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v15/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1169111,BANDWIDTH=1256754,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=768x432,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v13/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=859838,BANDWIDTH=932269,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=640x360,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v12/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=552728,BANDWIDTH=579426,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=480x270,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v11/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=345440,BANDWIDTH=371212,CODECS="hvc1.2.4.L123.B0,ec-3",RESOLUTION=416x234,FRAME-RATE=30.000,CLOSED-CAPTIONS="cc",AUDIO="a3",SUBTITLES="sub1"
v10/prog_index.m3u8

Accepted Reply

The playlist is correct. A video rendition group should only be used when the variant is different content. (For example, an different camera angle.)


In the case where the client is unable to play certain variants those should be removed from the active list.


When playing a particular variant the client should stay with that variant until it decides to do a bit rate switch.

When the switch occurs we may switch video codec but we try to avoid doing so.


Imagine you had only AVC codecs below some bit rate and a mix of AVC and HEVC above. If we had begun playing HEVC at a high bit rate and had to switch down into the range where there are only AVC we want to allow the switch to occur. The codec switch doesn't cause a problem for our players because we always instantiate a new decoder to handle the variant we switch to.

Replies

The playlist is correct. A video rendition group should only be used when the variant is different content. (For example, an different camera angle.)


In the case where the client is unable to play certain variants those should be removed from the active list.


When playing a particular variant the client should stay with that variant until it decides to do a bit rate switch.

When the switch occurs we may switch video codec but we try to avoid doing so.


Imagine you had only AVC codecs below some bit rate and a mix of AVC and HEVC above. If we had begun playing HEVC at a high bit rate and had to switch down into the range where there are only AVC we want to allow the switch to occur. The codec switch doesn't cause a problem for our players because we always instantiate a new decoder to handle the variant we switch to.

Thank for the information, it was helpful.