Create AVPlayer instance from Data (rather than URL)?

Question

Created Dec ’23

Replies 2

Boosts 0

Views 1.5k

Participants 3

In my SwiftUI/SwiftData application, I want to store videos in SwiftData objects (using external storage). To display them, I need to instantiate an AVPlayer (for use in a VideoPlayer view). But AVPlayer expects a URL, not a Data object.

Obviously, I can solve this problem via a file-caching scheme (i.e., by creating a local file when needed, using an LRU cache to control it's lifetime), but this results in an extra copy of the data (besides the hidden local file managed by SwiftData/CoreData). However, since videos can be quite large, I would prefer not to do that.

Has anyone any thoughts about how I can avoid the extra data copy?

Boost

Answer 1

DTS Engineer OP

Apple

Dec ’23

In this situation, your best option might be to use AVSampleBufferDisplayLayer and its friends:

https://developer.apple.com/documentation/avfoundation/avsamplebufferdisplaylayer/

There's no sample code for video, but there is for the audio side of this API set:

https://developer.apple.com/documentation/avfaudio/audio_engine/playing_custom_audio_with_your_own_player

Looking at this at a slightly higher level, it'd be unreasonable to play anything but the smallest videos from an in-memory Data object. (This would not be true of a memory-mapped Data object, but In that case you're basically have a URL anyway.) Even with AVSampleBufferDisplayLayer you'd be likely to load the data in smaller segments. That in turn will raise the question of whether SwiftData is the proper container for this data.

0

Answer 2

Dirk-FU OP

Dec ’23

I came up with the following solution to dynamically load Assets from Data after reading a thread on StackOverflow :

import Foundation
import AVFoundation

#if os(macOS)
import AppKit
#else
import UIKit
#endif

// Don't forget to set the File Type field in the asset catalog to the correct UTI string: com.apple.quicktime-movie
extension NSDataAsset:AVAssetResourceLoaderDelegate{
    @objc public func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
        
        if let infoRequest = loadingRequest.contentInformationRequest{
            infoRequest.isEntireLengthAvailableOnDemand = true
            infoRequest.contentType = typeIdentifier
            infoRequest.contentLength = Int64(data.count)
            infoRequest.isByteRangeAccessSupported = true
#if DEBUG
                print(infoRequest)
#endif
        }
        
        if let dataRequest = loadingRequest.dataRequest{
#if DEBUG
            print(dataRequest)
#endif
            dataRequest.respond(with: data.subdata(in:Int(dataRequest.requestedOffset) ..< Int(dataRequest.requestedOffset) + dataRequest.requestedLength))
            loadingRequest.finishLoading()
            
            return true
        }
        return false
    }
}
extension AVURLAsset{
    public convenience init?(_ dataAsset:NSDataAsset){
        guard let name = dataAsset.name.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed),
              let url = URL(string:"NSDataAsset://\(name))")
        else {return nil}
        
        self.init(url:url) // not really used!
        self.resourceLoader.setDelegate(dataAsset, queue: .main)
        // Retain the weak delegate for the lifetime of AVURLAsset
        objc_setAssociatedObject(self, "AVURLAsset+NSDataAsset", dataAsset, .OBJC_ASSOCIATION_RETAIN)
    }
}

0