I have run into several questions regarding mediastreamvalidator (see my other posts) but haven't been able to find anyone who can answer them. I think I could answer my own questions if I had the source code for the tool. Is it available? Seems like it would make sense to make it public to help support the HLS standard's adoption.
Thanks,
Phil
I have a lot of problems with this tool as well. It's extremely misleading and doesn't really help you align yourself any closer to the latest hls authoring guidelines. What I find even more annoying is that Apple's official examples, although, a bit contrived, all have "Must fix issues" and a handful of "Should fix issues". Apple's hls examples should not return any validation issues with their own tool. I mean what's up with their IDRs? Do they know something we don't know? They should really just pull this tool until their own streams can pass. It's great for pulling the metadata into a formatted table, but if they can't follow their own spec themselves this tool just sends developers down an unnecessary rabbit hole
It would be nice if they also added a bit more verbosity to their issues, even link back to their own guidelines doc.
Apple's official examples
https://developer.apple.com/streaming/examples/
HLS Validation Report
Stream type: VOD
https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_adv_example_hevc/master.m3u8
Variant Overview
| a1 | 1 | 339 | 0.0% | 304 | 0.0% | 416 x 234 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 2 | 583 | 0.0% | 533 | 0.0% | 480 x 270 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 3 | 1018 | 0.0% | 891 | 0.0% | 640 x 360 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 4 | 1476 | 0.0% | 1278 | 0.0% | 768 x 432 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 5 | 2524 | 0.0% | 2191 | 0.0% | 960 x 540 | 0.342 | 60.000 | AVC | High | 3.2 | - | 100% |
| ↳ | 6 | 3611 | 0.0% | 3184 | 0.0% | 1280 x 720 | 0.342 | 60.000 | AVC | High | 3.2 | - | 100% |
| ↳ | 7 | 5328 | -1.7% | 4650 | -0.7% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 93% |
| ↳ | 8 | 6737 | -7.9% | 5971 | -2.6% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 78% |
| ↳ | 9 | 8686 | -12.0% | 7759 | -3.6% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 67% |
| a2 | 10 | 562 | 0.0% | 526 | 0.0% | 416 x 234 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 11 | 805 | 0.0% | 756 | 0.0% | 480 x 270 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 12 | 1240 | 0.0% | 1113 | 0.0% | 640 x 360 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 13 | 1698 | 0.0% | 1500 | 0.0% | 768 x 432 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 14 | 2746 | 0.0% | 2413 | 0.0% | 960 x 540 | 0.342 | 60.000 | AVC | High | 3.2 | - | 100% |
| ↳ | 15 | 3834 | 0.0% | 3406 | 0.0% | 1280 x 720 | 0.342 | 60.000 | AVC | High | 3.2 | - | 100% |
| ↳ | 16 | 5551 | -1.7% | 4873 | -0.6% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 93% |
| ↳ | 17 | 6960 | -7.7% | 6194 | -2.5% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 78% |
| ↳ | 18 | 8909 | -11.8% | 7982 | -3.5% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 67% |
| a3 | 19 | 370 | 0.0% | 334 | 0.0% | 416 x 234 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 20 | 613 | 0.0% | 564 | 0.0% | 480 x 270 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 21 | 1048 | 0.0% | 921 | 0.0% | 640 x 360 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 22 | 1506 | 0.0% | 1308 | 0.0% | 768 x 432 | 1.253 | 30.000 | AVC | High | 3.1 | - | 100% |
| ↳ | 23 | 2554 | 0.0% | 2221 | 0.0% | 960 x 540 | 0.342 | 60.000 | AVC | High | 3.2 | - | 100% |
| ↳ | 24 | 3642 | 0.0% | 3214 | 0.0% | 1280 x 720 | 0.342 | 60.000 | AVC | High | 3.2 | - | 100% |
| ↳ | 25 | 5359 | -1.7% | 4681 | -0.7% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 93% |
| ↳ | 26 | 6768 | -7.9% | 6002 | -2.6% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 78% |
| ↳ | 27 | 8717 | -12.0% | 7790 | -3.6% | 1920 x 1080 | 0.340 | 60.000 | AVC | High | 4.2 | - | 67% |
| a1 | 28 | 341 | 0.0% | 315 | 0.0% | 416 x 234 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 29 | 549 | 0.0% | 522 | 0.0% | 480 x 270 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 30 | 902 | 0.0% | 829 | 0.0% | 640 x 360 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 31 | 1226 | 0.0% | 1139 | 0.0% | 768 x 432 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 32 | 2164 | 0.0% | 1966 | 0.0% | 960 x 540 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 33 | 2765 | 0.0% | 2636 | 0.0% | 1280 x 720 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 34 | 4080 | 0.0% | 3441 | 0.0% | 1920 x 1080 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 35 | 5192 | -4.3% | 4773 | -0.6% | 1920 x 1080 | 2.627 | 60.000 | HEVC | Main 10 | 4.1 | - | 92% |
| ↳ | 36 | 6664 | 0.0% | 6127 | 0.4% | 1920 x 1080 | 2.627 | 60.000 | HEVC | Main 10 | 4.1 | - | 79% |
| a2 | 37 | 563 | 0.0% | 537 | 0.0% | 416 x 234 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 38 | 771 | 0.0% | 745 | 0.0% | 480 x 270 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 39 | 1124 | 0.0% | 1052 | 0.0% | 640 x 360 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 40 | 1449 | 0.0% | 1361 | 0.0% | 768 x 432 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 41 | 2387 | 0.0% | 2189 | 0.0% | 960 x 540 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 42 | 2987 | 0.0% | 2858 | 0.0% | 1280 x 720 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 43 | 4302 | 0.0% | 3664 | 0.0% | 1920 x 1080 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 44 | 5414 | -4.2% | 4995 | -0.6% | 1920 x 1080 | 2.627 | 60.000 | HEVC | Main 10 | 4.1 | - | 92% |
| ↳ | 45 | 6887 | 0.0% | 6350 | 0.4% | 1920 x 1080 | 2.627 | 60.000 | HEVC | Main 10 | 4.1 | - | 79% |
| a3 | 46 | 371 | 0.0% | 345 | 0.0% | 416 x 234 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 47 | 579 | 0.0% | 553 | 0.0% | 480 x 270 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 48 | 932 | 0.0% | 860 | 0.0% | 640 x 360 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 49 | 1257 | 0.0% | 1169 | 0.0% | 768 x 432 | 2.505 | 30.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 50 | 2195 | 0.0% | 1997 | 0.0% | 960 x 540 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 51 | 2795 | 0.0% | 2666 | 0.0% | 1280 x 720 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 52 | 4110 | 0.0% | 3472 | 0.0% | 1920 x 1080 | 2.629 | 60.000 | HEVC | Main 10 | 4.1 | - | 100% |
| ↳ | 53 | 5222 | -4.3% | 4803 | -0.6% | 1920 x 1080 | 2.627 | 60.000 | HEVC | Main 10 | 4.1 | - | 92% |
| ↳ | 54 | 6695 | 0.0% | 6158 | 0.4% | 1920 x 1080 | 2.627 | 60.000 | HEVC | Main 10 | 4.1 | - | 79% |
Average duration processed: 0:10:00
Rendition Overview
| a1 | 55 | 162 | 162 | Audio | English | - | - | AAC-LC | Stereo | 48000 | en-US | - | 100% |
| a2 | 56 | 384 | 384 | Audio | English | - | - | AC-3 | 5.1 | 48000 | en-US | - | 100% |
| a3 | 57 | 192 | 192 | Audio | English | - | - | EC-3 | 5.1 | 48000 | en-US | - | 100% |
| cc | 58 | - | - | CC | English | CC1 | - | - | - | - | en | - | N/A |
| sub1 | 59 | 0 | 0 | Subtitle | English | - | WebVTT | - | - | - | en | - | 100% |
Average duration processed: 0:10:00
I-Frame Variant Overview
| 60 | 94 | -1.8% | 91 | -0.1% | 363 | 4x | 480 x 270 | 0.500 | 0.000 | AVC | High | 3.1 | - | 14% |
| 61 | 187 | -0.1% | 177 | 0.0% | 708 | 4x | 640 x 360 | 0.500 | 0.000 | AVC | High | 3.1 | - | 14% |
| 62 | 520 | -0.1% | 509 | 0.0% | 2036 | 4x | 960 x 540 | 0.500 | 0.000 | AVC | High | 3.1 | - | 14% |
| 63 | 760 | 0.0% | 732 | 0.0% | 2927 | 4x | 1280 x 720 | 0.500 | 0.000 | AVC | High | 3.1 | - | 14% |
| 64 | 1016 | 0.0% | 928 | 0.0% | 3712 | 4x | 1920 x 1080 | 0.500 | 0.000 | AVC | High | 4 | - | 14% |
| 65 | 52 | -0.1% | 51 | 0.0% | 404 | 8x | 480 x 270 | 1.000 | 0.000 | HEVC | Main 10 | 4.1 | - | 29% |
| 66 | 93 | -0.1% | 91 | 0.0% | 727 | 8x | 640 x 360 | 1.000 | 0.000 | HEVC | Main 10 | 4.1 | - | 29% |
| 67 | 159 | -0.1% | 154 | -0.1% | 1231 | 8x | 960 x 540 | 1.000 | 0.000 | HEVC | Main 10 | 4.1 | - | 29% |
| 68 | 226 | -0.1% | 217 | 0.0% | 1732 | 8x | 1280 x 720 | 1.000 | 0.000 | HEVC | Main 10 | 4.1 | - | 29% |
| 69 | 328 | -0.1% | 287 | -0.1% | 2295 | 8x | 1920 x 1080 | 1.000 | 0.000 | HEVC | Main 10 | 4.1 | - | 29% |
Average duration processed: 0:10:00
Checked against HLS Authoring Specification for Apple Devices
Must Fix Issues
1. The server MUST deliver playlists using gzip content-encoding
2. Measured peak bitrate compared to master playlist declared value exceeds error tolerance
Should Fix Issues
3. Target durations SHOULD be 6 seconds.
4. Content protection SHOULD follow the FairPlay Streaming (FPS) specification
5. You SHOULD support stream failover
6. Key frames (IDRs) SHOULD be present every two seconds
Report Information
| MediaStreamValidator version: | Version 1.2 (373.7-180828) |
| Report generator version: | 1.2 (373.7-180828) |
| Report generation date: | 11/07/18 10:40:11 |
| JSON format version: | 1.1 |
| JSON data timestamp: | 11/07/18 10:39:51 |
Thanks for the feedback, but the most effective place to give feeedback is via a radar or a support incident.
Concerning medaistreamvalidtor source. Nope, not available. Not likely to made available either.
As far as the samples having issues, there are several reasons for that:
- Samples may have passed the validator when the were put up, and the validator got more strict or modified rules later. (But yes, we should probably regen them to be more in line with the current version.)
- The issue of GZIP encoding is really only for production strreams which the sample is not.
- "Should fix" errors are things that you can violate. For example, no stream failover, or not using FPS.
