How to implement resource loader for linear/live assets without altering default behavior?

In our tvOS app we have to inject some tiny bit of data in the master manifest and leave the rest as is. The idea I was trying to implement here is intercepting the master manifest request with use of AVAssetResourceLoaderDelegate, and just redirect all consequent request, so AVKit can handle it on its own. In order to actually mimic the original requests, I made a copy of what is in AVAssetResourceLoadingRequest and adjusted only the parts required:

override func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
    guard let url = loadingRequest.request.url, url.scheme == Self.assetScheme else {
        return super.resourceLoader(resourceLoader, shouldWaitForLoadingOfRequestedResource: loadingRequest)
    }

    guard var urlComponents = URLComponents(url: url, resolvingAgainstBaseURL: false) else {
        LOG.error("Could not obtain url components from resource request: \(loadingRequest.request)")
        return super.resourceLoader(resourceLoader, shouldWaitForLoadingOfRequestedResource: loadingRequest)
    }
    urlComponents.scheme = "https"

    guard let assetURL = try? urlComponents.asURL() else {
        LOG.error("Could not make url from URL components \(urlComponents)")
        return super.resourceLoader(resourceLoader, shouldWaitForLoadingOfRequestedResource: loadingRequest)
    }

    let assetURLRequest = (loadingRequest.request as NSURLRequest).mutableCopy() as? NSMutableURLRequest
    assetURLRequest?.url = assetURL
    guard let taskRequest = assetURLRequest?.copy() as? URLRequest else {
        LOG.error("Could not convert url request \(String(describing: assetURLRequest))")
        return super.resourceLoader(resourceLoader, shouldWaitForLoadingOfRequestedResource: loadingRequest)
    }

    if url == masterManifestURL {

        // ...custom logic comes here...

    } else {
        loadingRequest.response = HTTPURLResponse(url: assetURL, statusCode: 302, httpVersion: "HTTP/1.1", headerFields: nil)
        loadingRequest.redirect = taskRequest
        loadingRequest.finishLoading()
    }
    return true
}

That works just fine, but only for VOD assets. For linear/live assets however only first bunch of data is loaded, when it ends, player does not request the next part of the sliding window and it hangs loading. I believe that the problem is somewhere with the custom logic, so I decided to list it separately:

urlSession.dataTask(with: taskRequest) { [weak self] data, response, error in
    loadingRequest.response = response
    if let data = data, let dataRequest = loadingRequest.dataRequest, let self = self, let manifestString =

String(data: data, encoding: .utf8) {
        let adjustedManifestString = self.adjustAudioMetadataForManifest(manifestString)
        if let adjustedData = adjustedManifestString.data(using: .utf8) {
            dataRequest.respond(with: adjustedData)
        } else {
            LOG.error("Could not complement audio labels in master manifest")
            dataRequest.respond(with: data)
        }
    }

    if let error = error {
        loadingRequest.finishLoading(with: error)
    } else {
        loadingRequest.finishLoading()
    }
}.resume()

I noticed that unlike apple player, the custom resource loader requests have different encoding headers. It also was not clear whether data length and offset is more crucial for linear than it is for VOD, so I added this header as well:

if let dataReq = loadingRequest.dataRequest, !dataReq.requestsAllDataToEndOfResource {
    let offsetEnd = dataRequest.requestedOffset + dataRequest.requestedLength - 1
    assetURLRequest?.addValue("bytes=\(dataReq.requestedOffset)-\(offsetEnd)", forHTTPHeaderField: "Range")
}

if loadingRequest.contentInformationRequest != nil {
    assetURLRequest?.setValue("identity", forHTTPHeaderField: "Accept-Encoding")
}

I also fulfilled contentInformationRequest which I forgot originally (but it still worked for VOD):

if let contentInformationRequest = loadingRequest.contentInformationRequest {
    contentInformationRequest.contentLength = Int64.max

    if let mimeType = response?.mimeType {
        let utiType = UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, mimeType as CFString,
nil)
        contentInformationRequest.contentType = utiType?.takeRetainedValue() as? String
    }
    contentInformationRequest.isByteRangeAccessSupported = true
}

and finally adjusted the data response with requested length:

if dataRequest.requestsAllDataToEndOfResource {
    dataRequest.respond(with: dataToRespond)
} else {
    let offsetStart = Int(dataRequest.requestedOffset)
    let offsetEnd = Int(dataRequest.requestedOffset + dataRequest.requestedLength)
    dataRequest.respond(with: dataToRespond[offsetStart ..< offsetEnd])
}

All those adjustments happen only for master manifest request, and I just redirect all other request to proper url with https schema. Unfortunately all adjustments don't seem to make any difference. All work equally good with VOD assets but doesn't allow linear assets to play beyond the the very first video record (so sliding window just hangs)

Is there some documentation on how to properly do the custom resource loader for a linear/live asset I can refer to in order to make it work?

Answered by Aleksandr Medvedev in 714610022

I've finally made it work with the trick I gave in the previous message. However I had to apply the renewal date not just for master manifest request, but also for all child manifests. It's unclear why "default" playback session cannot do it itself when redirecting child manifest requests, perhaps it could not make an educated guess about type of the asset (VOD/live) without having the information from master manifest, but my implementation doesn't rely on it either:

if let httpResonse = response as? HTTPURLResponse, let expirationValue = httpResonse.value(forHTTPHeaderField: "Expires") {
    let dateFormatter = DateFormatter()
    dateFormatter.locale = Locale(identifier: "en_US_POSIX")
    dateFormatter.dateFormat = "EEE, dd MMM yyyy HH:mm:ss zzz"
    if let expirationDate = dateFormatter.date(from: expirationValue) {
        let renewDate = max(expirationDate, Date(timeIntervalSinceNow: 8))
        contentInformationRequest.renewalDate = renewDate
    }
}

This line let renewDate = max(expirationDate, Date(timeIntervalSinceNow: 8)) adds 8 seconds grace period for the player to load videos. Otherwise it does not keep up with the pace of renewals, and video loads in poor quality.

With use of http-proxy I looked into the sequence of http requests AVPlayer makes by default for live assets (without having a AVAssetResourceLoaderDelegate set for the asset), and spotted that it makes requests with fixed delay (or it least it seems like that). Apparently renewalDate of AVAssetResourceLoadingContentInformationRequest instance should do the job, however after playing around with it I could not make it work with any values (whether it's a few seconds in future, nearing current time or set in the past) i.e. func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForRenewalOfRequestedResource renewalRequest: AVAssetResourceRenewalRequest) -> Bool method of the delegate does not get called. Another thing which drew my attention is that for live assets all manifests (master and child) have "Expires" http header set to the time of the request (that's probably what AVPlayer itself uses to deduce that the current asset is live and needs to be constantly re-requested), so ended up setting this date for the the master manifest's content information, but it didn't fix the issue either:

if let httpResonse = response as? HTTPURLResponse, let expirationValue = httpResonse.value(forHTTPHeaderField: "Expires") {
    let dateFormatter = DateFormatter()
    dateFormatter.locale = Locale(identifier: "en_US_POSIX")
    dateFormatter.dateFormat = "EEE, dd MMM yyyy HH:mm:ss zzz"
    contentInformationRequest.renewalDate = dateFormatter.date(from: expirationValue)
}

For some reason comprehensive documentation which would explain why it doesn't work or any sample codes with use of this property don't exist in my google. Any idea or reference I can refer to in regards to the renewalDate property?

Accepted Answer

I've finally made it work with the trick I gave in the previous message. However I had to apply the renewal date not just for master manifest request, but also for all child manifests. It's unclear why "default" playback session cannot do it itself when redirecting child manifest requests, perhaps it could not make an educated guess about type of the asset (VOD/live) without having the information from master manifest, but my implementation doesn't rely on it either:

if let httpResonse = response as? HTTPURLResponse, let expirationValue = httpResonse.value(forHTTPHeaderField: "Expires") {
    let dateFormatter = DateFormatter()
    dateFormatter.locale = Locale(identifier: "en_US_POSIX")
    dateFormatter.dateFormat = "EEE, dd MMM yyyy HH:mm:ss zzz"
    if let expirationDate = dateFormatter.date(from: expirationValue) {
        let renewDate = max(expirationDate, Date(timeIntervalSinceNow: 8))
        contentInformationRequest.renewalDate = renewDate
    }
}

This line let renewDate = max(expirationDate, Date(timeIntervalSinceNow: 8)) adds 8 seconds grace period for the player to load videos. Otherwise it does not keep up with the pace of renewals, and video loads in poor quality.

@Aleksandr Medvedev , how to compute dataToRespond?

I am trying to play a video with AVPlayer but want to support self signed certificates and need to use the AVAssetResourceLoaderDelegate. I am getting the callback, but the video is not playing.

Of course, I did not use self.adjustAudioMetadataForManifest as it seems something specific to your need.

How to implement resource loader for linear/live assets without altering default behavior?
 
 
Q