We're experiencing significant issues with AVPlayer when attempting to play partially downloaded HLS content in offline mode. Our app downloads HLS video content for offline viewing, but users encounter the following problems:
Excessive Loading Delay: When offline, AVPlayer attempts to load resources for up to 60 seconds before playing the locally available segments
Asset Loss: Sometimes AVPlayer completely loses the asset reference and fails to play the video on subsequent attempts
Inconsistent Behavior: The same partially downloaded asset might play immediately in one session but take 30+ seconds in another
Network Activity Despite Offline Settings: Despite configuring options to prevent network usage, AVPlayer still appears to be attempting network connections
These issues severely impact our offline user experience, especially for users with intermittent connectivity.
Technical Details
Implementation Context
Our app downloads HLS videos for offline viewing using AVAssetDownloadTask. We store the downloaded content locally and maintain a dictionary mapping of file identifiers to local paths. When attempting to play these videos offline, we experience the described issues.
Current Implementation
Here's our current implementation for playing the videos:
- (void)presentNativeAvplayerForVideo:(Video *)video navContext:(NavContext *)context {
NSString *localPath = video.localHlsPath;
if (localPath) {
NSURL *videoURL = [NSURL URLWithString:localPath];
NSDictionary *options = @{
AVURLAssetPreferPreciseDurationAndTimingKey: @YES,
AVURLAssetAllowsCellularAccessKey: @NO,
AVURLAssetAllowsExpensiveNetworkAccessKey: @NO,
AVURLAssetAllowsConstrainedNetworkAccessKey: @NO
};
AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:videoURL options:options];
AVPlayerViewController *playerViewController = [[AVPlayerViewController alloc] init];
NSArray *keys = @[@"duration", @"tracks"];
[asset loadValuesAsynchronouslyForKeys:keys completionHandler:^{
dispatch_async(dispatch_get_main_queue(), ^{
AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset];
AVPlayer *player = [AVPlayer playerWithPlayerItem:playerItem];
playerViewController.player = player;
[player play];
});
}];
playerViewController.modalPresentationStyle = UIModalPresentationFullScreen;
[context presentViewController:playerViewController animated:YES completion:nil];
}
}
Attempted Solutions
We've tried several approaches to mitigate these issues:
Modified Asset Options:
NSDictionary *options = @{
AVURLAssetPreferPreciseDurationAndTimingKey: @NO, // Changed to NO
AVURLAssetAllowsCellularAccessKey: @NO,
AVURLAssetAllowsExpensiveNetworkAccessKey: @NO,
AVURLAssetAllowsConstrainedNetworkAccessKey: @NO,
AVAssetReferenceRestrictionsKey: @(AVAssetReferenceRestrictionForbidRemoteReferenceToLocal)
};
Skipped Asynchronous Key Loading:
AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset automaticallyLoadedAssetKeys:nil];
Modified Player Settings:
player.automaticallyWaitsToMinimizeStalling = NO;
[playerItem setPreferredForwardBufferDuration:2.0];
Added Network Resource Restrictions:
playerItem.canUseNetworkResourcesForLiveStreamingWhilePaused = NO;
Used File URLs Instead of HTTP URLs where possible
Despite these attempts, the issues persist.
Expected vs. Actual Behavior
Expected Behavior:
AVPlayer should immediately begin playback of locally available HLS segments
When offline, it should not attempt to load from network for more than a few seconds
Once an asset is successfully played, it should be reliably available for future playback
Actual Behavior:
AVPlayer waits 10-60 seconds before playing locally available segments
Network activity is observed despite all network-restricting options
Sometimes the player fails completely to play a previously available asset
Behavior is inconsistent between playback attempts with the same asset
Questions:
What is the recommended approach for playing partially downloaded HLS content offline with minimal delay?
Is there a way to force AVPlayer to immediately use available local segments without attempting to load from the network?
Are there any known issues with AVPlayer losing references to locally stored HLS assets?
What diagnostic steps would you recommend to track down the specific cause of these delays?
Does AVFoundation have specific timeouts for offline HLS playback that could be configured?
Any guidance would be greatly appreciated as this issue is significantly impacting our user experience.
Device Information
iOS Versions Tested: 14.5 - 18.1
Device Models: iPhone 12, iPhone 13, iPhone 14, iPhone 15
Xcode Version: 15.3-16.2.1
HTTP Live Streaming
RSS for tagSend audio and video over HTTP from an ordinary web server for playback on Mac, iOS, and tvOS devices using HTTP Live Streaming (HLS).
Posts under HTTP Live Streaming tag
76 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I am playing the protected HLS streams and the authorization token expires in 3 minutes. I am trying to achieve this with 'AVAssetResourceLoaderDelegate'. I can refresh the token and play it, but the problem is in between the session, the player stalls for a small time, LIKE 1 SECOND.
Here's my code :
class APLCustomAVARLDelegate: NSObject, AVAssetResourceLoaderDelegate {
static let httpsScheme = "https"
static let redirectErrorCode = 302
static let badRequestErrorCode = 400
private var token: String?
private var retryDictionary = [String: Int]()
private let maxRetries = 3
private func schemeSupported(_ scheme: String) -> Bool {
let supported = ishttpSchemeValid(scheme)
print("Scheme '\(scheme)' supported: \(supported)")
return supported
}
private func reportError(loadingRequest: AVAssetResourceLoadingRequest, error: Int) {
let nsError = NSError(domain: NSURLErrorDomain, code: error, userInfo: nil)
print("Reporting error: \(nsError)")
loadingRequest.finishLoading(with: nsError)
}
// Handle token renewal requests to prevent playback stalls
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForRenewalOfRequestedResource renewalRequest: AVAssetResourceRenewalRequest) -> Bool {
print("Resource renewal requested for URL: \(renewalRequest.request.url?.absoluteString ?? "unknown URL")")
// Handle renewal the same way we handle initial requests
guard let scheme = renewalRequest.request.url?.scheme else {
print("No scheme found in the renewal URL.")
return false
}
if isHttpsSchemeValid(scheme) {
return handleHttpsRequest(renewalRequest)
}
print("Scheme not supported for renewal.")
return false
}
private func isHttpsSchemeValid(_ scheme: String) -> Bool {
let isValid = scheme == APLCustomAVARLDelegate.httpsScheme
print("httpsScheme scheme '\(scheme)' valid: \(isValid)")
return isValid
}
private func generateHttpsURL(sourceURL: URL) -> URL? {
// If you need to modify the URL, do it here
// Currently this just returns the same URL
let urlString = sourceURL.absoluteString
print("Generated HTTPS URL: \(urlString)")
return URL(string: urlString)
}
private func handleHttpsRequest(_ loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
print("Handling HTTPS request.")
guard let sourceURL = loadingRequest.request.url,
var redirectURL = generateHttpsURL(sourceURL: sourceURL) else {
print("Failed to generate HTTPS URL.")
reportError(loadingRequest: loadingRequest, error: APLCustomAVARLDelegate.badRequestErrorCode)
return true
}
// Track retry attempts with a dictionary keyed by request URL
let urlString = sourceURL.absoluteString
let currentRetries = retryDictionary[urlString] ?? 0
if currentRetries < maxRetries {
retryDictionary[urlString] = currentRetries + 1
} else {
// Too many retries, report a more specific error
reportError(loadingRequest: loadingRequest, error: NSURLErrorTimedOut)
retryDictionary.removeValue(forKey: urlString)
return true
}
if var urlComponents = URLComponents(url: redirectURL, resolvingAgainstBaseURL: false) {
var queryItems = urlComponents.queryItems ?? []
// Generate a fresh token each time
let freshToken = AESTimeBaseEncription.secureEncryptSecretText()
// Check if the token already exists
if let existingTokenIndex = queryItems.firstIndex(where: { $0.name == "token" }) {
// Update the existing token
queryItems[existingTokenIndex].value = freshToken
} else {
// Add the token if it doesn't exist
queryItems.append(URLQueryItem(name: "token", value: freshToken))
}
urlComponents.queryItems = queryItems
redirectURL = urlComponents.url!
}
let redirectRequest = URLRequest(url: redirectURL)
let response = HTTPURLResponse(url: redirectURL, statusCode: APLCustomAVARLDelegate.redirectErrorCode, httpVersion: nil, headerFields: nil)
print("Redirecting HTTPS to URL: \(redirectURL)")
loadingRequest.redirect = redirectRequest
loadingRequest.response = response
loadingRequest.finishLoading()
// If successful, reset the retry counter
if retryDictionary[urlString] == maxRetries {
retryDictionary.removeValue(forKey: urlString)
}
return true
}
}
I can't play video content with HEVC and DRM. Tested HEVC only: OK. Tested DRM+AVC: Ok.
Tested 2 players (Clappr/Stevie and BitMovin)
Master, variants and EXT-X-MAPs are downloaded Ok, DRM keys Ok and then, for instance with BitMovin Player:
[BMP] [Player] [Error] Event: SourceError, Data: {"code":2001,"data":{"message":"The operation couldn’t be completed. (CoreMediaErrorDomain error -12927.)","code":-12927},"message":"Source Error. The operation couldn’t be completed. (CoreMediaErrorDomain error -12927.)","timestamp":1740320663.4505711,"type":"onSourceError"} code: 2001 [Data code: -12927, message: The operation couldn’t be completed. (CoreMediaErrorDomain error -12927.), underlying error: Error Domain=CoreMediaErrorDomain Code=-12927 "(null)"]
4k-master.m3u8.txt
4k.m3u8.txt
4k-audio.m3u8.txt
Hello,
Our users have started to see a new fatal AVPlayer error during playback starting with iOS/tvOS 18.0. The error is defined as "CoreMediaErrorDomain Code=-15486".
We have not been able to reproduce this issue locally within our development team.
Is there any documentation on the cause of this error or steps to recover from this error?
Thank you,
Howard
Hello, Apple video engineers.
According to the official documentation, HLS is built on HTTP and traditionally ran on top of TCP. However, with the introduction of HTTP/3, which uses QUIC (runs on top of UDP), I would like to clarify the following:
Has the official HLS specification changed in a way that allows it to be considered UDP-based when using HTTP/3? And is it fair to say that HLS supports UDP since the transport can go over HTTP/3 and QUIC?
Would it be more accurate to say that HLS remains HTTP-dependent, and the transport protocol (TCP or QUIC) only determines how HTTP requests are delivered?
My thoughts: Since HTTP/3 uses QUIC running over UDP, we still can't say that HLS supports UDP in a classical way, as it is introduced in RTP, RTSP, SRT.
I am using videojs player to play my hls m3u8 proxies, and there was no issue with ipad os 16, ater i upgraded it to 18, i started getting this warning ->
[Warning] VIDEOJS: – "WARN:"
"Problem encountered with playlist 0-https://coludFront.m3u8. Trying again since it is the only playlist."
After this warning logs, the player just freeze's and even the source is not loaded into player.
This issue doesn't exist in mac safari and iphone safari with latest os.
I am developing an app to stream and download DRM protected HLS videos based on the official “FairPlay Streaming Server SDK”.
When I play the downloaded video, it asks the server for .ts or .aac, even though I have passed the path of the downloaded video to AVURLAsset.
As a result, playback fails when the device is offline, such as in airplane mode.
This behavior depends on the playback time of the video and occurs when trying to download and play a video with a playback time of 19 hours or more.
It did not occur for videos with a playback time of 18 hours.
The environment we checked is iOS 18.3.
The solution at this time is to limit the video playback time to 18 hours, but if possible, we would like to allow download playback of videos longer than 19 hours.
Does anyone have any information or know of a solution to this problem, such as if you have experienced this type of event, or if you know that content longer than 19 hours cannot be played offline?
// load
let path = ".../***.movpkg" // Path of the downloaded file
videoAsset = AVURLAsset(url: path)
playerItem = AVPlayerItem(asset: videoAsset!)
player.replaceCurrentItem(with: playerItem)
// isPlayableOffline
print("videoAsset.assetCache.isPlayableOffline = \(videoAsset.assetCache.isPlayableOffline)") // true
Hi folks,
When doing HLS v6 live streaming with fmp4 chunks we noticed that when the encoder timestamps slightly drift and a #EXT-X-DISCONTINUITY tag is created in either the audio or video playlist (in an ABR setup), the tag is not correctly handled by the player leading to a broken playback containing black screen or no audio (depending on which playlist the tag is printed in).
We noticed that this is often true when the number of tags is odd between the playlists (eg. the audio playlist contains 1 tag and the video contains 2 tags will result in a black screen with audio).
By using the same "broken" source but using Shaka player instead won't break the playback at all.
Are there any possible fix (or upcoming) for AV Player?
Hello, we have HLS Stream app on Apple TV. Our streams are DRM protected. We have problem with streams when source device is turned off. For example, user start to watch our HLS DRM Protected content. After some time, user turns off device (it can be Monitor or TV via connected HDMI). Our app does not understand HDMI Source device turned off. Is there any way to understand HDMI connected device is turned off on Swift?
In our Apple TV application, we use the native AVPlayer for live playback functionality. Until tvOS 17.6 and during the tvOS 18 beta, the Pause/Resume feature worked as expected, allowing us to pause live playback. However, after updating to tvOS 18.1, the pause functionality no longer works.
The same app still works fine on tvOS 17, but on tvOS 18, attempting to pause live playback has no effect. We reviewed the tvOS 18 release notes but couldn't find any relevant changes or deprecations related to AVPlayer or live playback behavior.
Has there been any change in the handling of live playback or the Pause/Resume functionality in tvOS 18.1? Any guidance or suggestions to address this issue would be greatly appreciated.
Thank you!
In our Apple TV application, we are using the native AVPlayer for live playback functionality. During live restart playback, we intermittently encounter an error when the playback timeline approaches the actual live event end time.
Error:
The operation couldn’t be completed. (CoreMediaErrorDomain error -16839 - Unable to get playlist before long download timer) / Failure reason:
Scenario:
The live event is scheduled from 7:00 AM to 8:00 AM.
Restart playback begins at 7:20 AM, allowing the user to watch the event from the start while the live stream continues in real-time.
As the restart playback timeline approaches the actual event end time (8:00 AM), AVPlayer displays an error, and playback continues in the background.
Hi there, we have found a problem, after switching audio tracks multi times when play the HLS, there will be a few seconds of no sound playing after audio track switching, Is there a solution to this problem?
Can some one please answer me How Can I update the cookies of the previously set m3u8 video in AVPlayer without creating the new AVURLAsset and replacing the AVPlayer current Item with it
Here we are focusing to change the cookie at every 120 seconds while playing , in apple avplayer we can't modify cookie after initialisation due to that we followed the approach to using " Resource loader delegate " to pass cookie as a header value .
What I notice is that the playlist file (.m3u8) gets downloaded correctly. Then video file (.m4a) some chunks also gets downloaded. I know that the .ts file is downloaded because I can see the GET request completing on the web server with status 200. I also set a breakpoint at the following line:
loadingRequest.dataRequest?.respond(with: data)
immediately got error from avplayer status as
"The operation could not be completed. An unknown error occurred (-12881) From core media"
Need confirmation on why I am unable to load HLS using resource loader.
is it possible to update cookie value while paying continuously on avplayer.
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
let urlString = "localhost://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/.m3u8"
guard let url = URL(string: urlString) else {
print("Invalid URL")
return
}
//Create cookie to prepare for player asset
let cookie = HTTPCookie(properties: [
.name: "dazn-token",
.value: "cookie value",
.domain: url.host() ?? "",
.path: "/",
.discard: true
])
//Create cookie key to set AVURLAsset
let options = [AVURLAssetHTTPCookiesKey: [cookie]]
let asset = AVURLAsset(url: url,options: options)
proxy = ReverseProxyResourceLoader()
proxy?.cookie = "exampleCookie"
// Set resource loader delegate to moniter the chunks
asset.resourceLoader.setDelegate(proxy, queue: DispatchQueue.global())
// Load asset keys asynchronously (e.g., "playable")
let keys = ["playable"]
// Initialize the AVPlayer with the URL
let playerItem = AVPlayerItem(asset: asset)
self.player = AVPlayer(playerItem: playerItem)
playerItem.addObserver(self, forKeyPath: "status", options: [.new, .initial], context: nil)
// Observe 'error' property (if needed)
playerItem.addObserver(self, forKeyPath: "error", options: [.new], context: nil)
let contentKeySessionDelegate = ContentKeyDelegate()
// Initialize AVContentKeySession
let contentKeySession = AVContentKeySession(keySystem: .clearKey)
self.contentKeySession = contentKeySession
contentKeySession.setDelegate(contentKeySessionDelegate, queue: DispatchQueue.main)
// Associate the asset with the content key session
contentKeySession.addContentKeyRecipient(asset)
// Create a layer for the AVPlayer and add it to the view
playerLayer = AVPlayerLayer(player: player)
playerLayer?.frame = view.bounds
playerLayer?.videoGravity = .resizeAspect
if let playerLayer = playerLayer {
view.layer.addSublayer(playerLayer)
}
NotificationCenter.default.addObserver(
self,
selector: #selector(playerDidFinishPlaying),
name: .AVPlayerItemDidPlayToEndTime,
object: player?.currentItem
)
// Start playback
player?.play()
}
// Update cookie when ever needed
func updateCookie() {
proxy?.cookie = "update exampleCookie"
}
@objc private func playerDidFinishPlaying(notification: Notification) {
print("Playback finished!")
// Optionally, handle end-of-playback actions here
}
//
// ReverseProxyResourceLoader.swift
// HLSDemo
//
// Created by Gajje.Venkatarao on 12/12/24.
//
import Foundation
import AVKit
import AVFoundation
class ReverseProxyResourceLoader: NSObject, AVAssetResourceLoaderDelegate {
var cookie = ""
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
resourceLoader.preloadsEligibleContentKeys = true
guard let interceptedURL = loadingRequest.request.url else {
loadingRequest.finishLoading(with: NSError(domain: "ReverseProxy", code: -1, userInfo: [NSLocalizedDescriptionKey: "Invalid URL"]))
return false
}
if interceptedURL.scheme == "skd" {
print("Token updated Cookie:", interceptedURL )
return false
}
var components = URLComponents(url: interceptedURL, resolvingAgainstBaseURL: false)
components?.scheme = "https" // Replace with the original scheme
guard let originalURL = components?.url else {
loadingRequest.finishLoading(with: NSError(domain: "ReverseProxy", code: -1, userInfo: [NSLocalizedDescriptionKey: "Failed to map URL"]))
loadingRequest.finishLoading()
return false
}
var request = URLRequest(url: originalURL)
request.httpMethod = "GET"
if let storeCoockie = HTTPCookie(properties: [
.name: "dazn-token",
.value: cookie,
.domain: originalURL.host ?? "",
.path: "/",
.discard: true
]){
HTTPCookieStorage.shared.setCookie(storeCoockie)
}
let headers = loadingRequest.request.allHTTPHeaderFields ?? [:]
for (key, value) in headers {
request.addValue(value, forHTTPHeaderField: key)
}
request.addValue(cookie, forHTTPHeaderField: "Cookie")
URLSession.shared.configuration.httpShouldSetCookies = true
request.httpShouldHandleCookies = true
let task = (URLSession.shared.dataTask(with: originalURL) { data, response, error in
if let error = error {
print("Error Received:", error)
loadingRequest.finishLoading(with: error)
return
}
print(originalURL)
guard let data = data , let url = response?.url else {
loadingRequest.finishLoading(with: NSError(domain: "ReverseProxy", code: -1, userInfo: [NSLocalizedDescriptionKey: "No data received"]))
return
}
loadingRequest.dataRequest?.respond(with: data)
loadingRequest.finishLoading()
} as URLSessionDataTask)
task.resume()
return true
}
}
Example project
I'm working on an iOS app using JWPlayer to stream an HLS video. However, the stream stops and doesn't restart on both emulators and real devices. The output provides the following information:
• transportType: HTTP Live Stream
• mediaType: HTTP Live Stream
• BundleID:
• name: MEDIA_PLAYBACK_STALL
• interfaceType: Wifi
Additionally, the log shows:
(16830): Media file not received in 21s
Can anyone guide me on this issue?
Is this technical solution reasonable about WKWebView on cross-domain issues ?
Hi,all
My project use WKWebView to load offline package, such as .html/.css/.js,and also request some resources from remote server to update pages. So there is a cross-domain problem with local file(file://***) and remote domain (https://***), is this following technical solution reasonable to fix this problem:
1. Create a custom URLSchemeHandler which conforms to WKURLSchemeHandler
2.Unify local file and remote domain request to https request
3. Hook WKWebView https request
4. Implement WKURLSchemeHandler delegate method
(void)webView:(WKWebView *)webView startURLSchemeTask:(id)urlSchemeTask {
NSURL *url = urlSchemeTask.request.URL;
if ([url.pathExtension isEqualToString:@"html"]) {
NSData *data = [[NSData alloc] initWithContentsOfFile:localFilePath];
NSMutableDictionary resHeader = [NSMutableDictionary new];
[resHeader setValue:@"" forKey:@"Access-Control-Allow-Origin"];
[resHeader setValue:@"charset=UTF-8" forKey:@"Content-Type"];
[resHeader setValue:@"text/html" forKey:@"Content-Type"];
NSHTTPURLResponse *response = [[NSHTTPURLResponse alloc]
initWithURL:url statusCode:200 HTTPVersion:@"HTTP/1.1" headerFields:resHeader];
[urlSchemeTask didReceiveResponse:response];
[urlSchemeTask didReceiveData:data];
[urlSchemeTask didFinish];
} else {
NSURLSession *defaultSession = [NSURLSession sharedSession];
NSURLSessionTask *dataTask = [defaultSession dataTaskWithRequest:urlSchemeTask.request completionHandler:^(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error) {
[urlSchemeTask didReceiveResponse:response];
[urlSchemeTask didReceiveData:data];
[urlSchemeTask didFinish];
}];
[dataTask resume];
}
}
Is this technical solution reasonable? and is there any issues that I haven't considered?
Sincerely,
Looking forward to your reply
Hello,
I'm writing a program to create CMAF compliant HLS files, with encryption.
I have a copy of ISO_IEC_23001-7_2023 to attempt to follow the spec.
I am following the 1:9 pattern encryption using CBCS, so for every 16 bytes of encrypted NAL unit data (of type 1 and 5), there's 144 bytes of clear data.
When testing my output in Safari with 'identity' keys Quickly Diagnosing Content Key and IV Issues, Safari will request the identity key from my test server and first few bytes of the CMAF renditions, but will not play and console gives away no clues to the error.
I am setting the subsample bytesofclear/protected data in the senc boxes. What I'm not sure of, is whether HLS/Safari/iOS acknowledges the senc/saiz/saio boxes of the MP4. There are other third party packagers Bento4, who suggest that they do not:
those clients ignore the explicit encryption
layout metadata found in saio/saiz boxes, and instead rely purely on the
video slice header size to determine the portions of the sample that is
encrypted
So now I'm fairly sure I need to decipher the video slice header size, and apply the protected blocks from that point on.
My question is, is that all there is to it? And is there a better way to debug my output? mediastreamvalidator will only work against unencrypted variants (which I'm outputting okay).
Thanks in advance!
Hi, I'm trying to decode a HLS livestream with VideoToolbox. The CMSampleBuffer is successfully created (OSStatus == noErr). When I enqueue the CMSampleBuffer to a AVSampleBufferDisplayLayer the view isn't displaying anything and the status of the AVSampleBufferDisplayLayer is 1 (rendering).
When I use a VTDecompressionSession to convert the CMSampleBuffer to a CVPixelBuffer the VTDecompressionOutputCallback returns a -8969 (bad data error).
What do I need to fix in my code? Do I incorrectly parse the data from the segment for the CMSampleBuffer?
let segmentData = try await downloadSegment(from: segment.url)
let (sps, pps, idr) = try parseH264FromTSSegment(tsData: segmentData)
if self.formatDescription == nil {
self.formatDescription = try CMFormatDescription(h264ParameterSets: [sps, pps])
}
if let sampleBuffer = try createSampleBuffer(from: idr, segment: segment) {
try self.decodeSampleBuffer(sampleBuffer)
}
func parseH264FromTSSegment(tsData: Data) throws -> (sps: Data, pps: Data, idr: Data) {
let tsSize = 188
var pesData = Data()
for i in stride(from: 0, to: tsData.count, by: tsSize) {
let tsPacket = tsData.subdata(in: i..<min(i + tsSize, tsData.count))
guard let payload = extractPayloadFromTSPacket(tsPacket) else { continue }
pesData.append(payload)
}
let nalUnits = parseNalUnits(from: pesData)
var sps: Data?
var pps: Data?
var idr: Data?
for nalUnit in nalUnits {
guard let firstByte = nalUnit.first else { continue }
let nalType = firstByte & 0x1F
switch nalType {
case 7: // SPS
sps = nalUnit
case 8: // PPS
pps = nalUnit
case 5: // IDR
idr = nalUnit
default:
break
}
if sps != nil, pps != nil, idr != nil {
break
}
}
guard let validSPS = sps, let validPPS = pps, let validIDR = idr else {
throw NSError()
}
return (validSPS, validPPS, validIDR)
}
func extractPayloadFromTSPacket(_ tsPacket: Data) -> Data? {
let syncByte: UInt8 = 0x47
guard tsPacket.count == 188, tsPacket[0] == syncByte else {
return nil
}
let payloadStart = (tsPacket[1] & 0x40) != 0
let adaptationFieldControl = (tsPacket[3] & 0x30) >> 4
var payloadOffset = 4
if adaptationFieldControl == 2 || adaptationFieldControl == 3 {
let adaptationFieldLength = Int(tsPacket[4])
payloadOffset += 1 + adaptationFieldLength
}
guard adaptationFieldControl == 1 || adaptationFieldControl == 3 else {
return nil
}
let payload = tsPacket.subdata(in: payloadOffset..<tsPacket.count)
return payloadStart ? payload : nil
}
func parseNalUnits(from h264Data: Data) -> [Data] {
let startCode = Data([0x00, 0x00, 0x00, 0x01])
var nalUnits: [Data] = []
var searchRange = h264Data.startIndex..<h264Data.endIndex
while let range = h264Data.range(of: startCode, options: [], in: searchRange) {
let nextStart = h264Data.range(of: startCode, options: [], in: range.upperBound..<h264Data.endIndex)?.lowerBound ?? h264Data.endIndex
let nalUnit = h264Data.subdata(in: range.upperBound..<nextStart)
nalUnits.append(nalUnit)
searchRange = nextStart..<h264Data.endIndex
}
return nalUnits
}
private func createSampleBuffer(from data: Data, segment: HLSSegment) throws -> CMSampleBuffer? {
var blockBuffer: CMBlockBuffer?
let alignedData = UnsafeMutableRawPointer.allocate(byteCount: data.count, alignment: MemoryLayout<UInt8>.alignment)
data.copyBytes(to: alignedData.assumingMemoryBound(to: UInt8.self), count: data.count)
let blockStatus = CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault,
memoryBlock: alignedData,
blockLength: data.count,
blockAllocator: nil,
customBlockSource: nil,
offsetToData: 0,
dataLength: data.count,
flags: 0,
blockBufferOut: &blockBuffer
)
guard blockStatus == kCMBlockBufferNoErr, let validBlockBuffer = blockBuffer else {
alignedData.deallocate()
throw NSError()
}
var sampleBuffer: CMSampleBuffer?
var timing = [calculateTiming(for: segment)]
var sampleSizes = [data.count]
let sampleStatus = CMSampleBufferCreate(
allocator: kCFAllocatorDefault,
dataBuffer: validBlockBuffer,
dataReady: true,
makeDataReadyCallback: nil,
refcon: nil,
formatDescription: formatDescription,
sampleCount: 1,
sampleTimingEntryCount: 1,
sampleTimingArray: &timing,
sampleSizeEntryCount: sampleSizes.count,
sampleSizeArray: &sampleSizes,
sampleBufferOut: &sampleBuffer
)
guard sampleStatus == noErr else {
alignedData.deallocate()
throw NSError()
}
return sampleBuffer
}
private func decodeSampleBuffer(_ sampleBuffer: CMSampleBuffer) throws {
guard let formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer) else {
throw NSError()
}
if decompressionSession == nil {
try setupDecompressionSession(formatDescription: formatDescription)
}
guard let session = decompressionSession else {
throw NSError()
}
let flags: VTDecodeFrameFlags = [._EnableAsynchronousDecompression, ._EnableTemporalProcessing]
var flagOut = VTDecodeInfoFlags()
let status = VTDecompressionSessionDecodeFrame(
session,
sampleBuffer: sampleBuffer,
flags: flags,
frameRefcon: nil,
infoFlagsOut: nil)
if status != noErr {
throw NSError()
}
}
private func setupDecompressionSession(formatDescription: CMFormatDescription) throws {
self.formatDescription = formatDescription
if let session = decompressionSession {
VTDecompressionSessionInvalidate(session)
self.decompressionSession = nil
}
var decompressionSession: VTDecompressionSession?
var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decompressionOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
let status = VTDecompressionSessionCreate(
allocator: kCFAllocatorDefault,
formatDescription: formatDescription,
decoderSpecification: nil,
imageBufferAttributes: nil,
outputCallback: &callback,
decompressionSessionOut: &decompressionSession
)
if status != noErr {
throw NSError()
}
self.decompressionSession = decompressionSession
}
let decompressionOutputCallback: VTDecompressionOutputCallback = { (
decompressionOutputRefCon,
sourceFrameRefCon,
status,
infoFlags,
imageBuffer,
presentationTimeStamp,
presentationDuration
) in
guard status == noErr else {
print("Callback: \(status)")
return
}
if let imageBuffer = imageBuffer {
}
}
In an m3u8 manifest, audio EXT-X-MEDIA tags usually contain CHANNELS tag containing the audio channels count like so:
#EXT-X-MEDIA:TYPE=AUDIO,URI="audio_clear_eng_stereo.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="stream_5",AUTOSELECT=YES,CHANNELS="2"
Is it possible to get this info from AVPlayer, AVMediaSelectionOption or some related API?
Our company application uses callkit for livestream but is not available in china which make Apple not to approve the app. can any one help with what i can do. The live stream is one of the key feature of the app. Or is it possible to restrict download from china