I have noticed that iOS 14, macOS 11, and tvOS 14 include the ability to process video files using a new VNVideoProcessor class. I have tried to leverage this within my code, in an attempt to perform a VNTrackObjectRequest, with no success. Specifically, my observations report invalid within the body, and the confidence and detected bounding box never change.
I am setting up my code like such;
A sample output I receive in the console during observation is;
OBSERVATION: <VNDetectedObjectObservation: 0x2827ee200> 032AB694-62E2-4674-B725-18EA2804A93F requestRevision=2 confidence=1.000000 timeRange={{0/90000 = 0.000}, {INVALID}} boundingBox=[0.333333, 0.138599, 0.162479, 0.207899]
I note that the observation reports something is invalid, alongside the fact that the confidence is always reported as 1.000000 and the bounding box coordinates never change. I'm unsure if this has to do with my lack of VNVideoProcessingOption setup or something else I am doing wrong.
I am setting up my code like such;
Code Block let videoProcessor = VNVideoProcessor(url: videoURL) let asset = AVAsset(url: videoURL) let completion: VNRequestCompletionHandler = { request, error in let observations = request.results as! [VNObservation] if let observation = observations.first as? VNDetectedObjectObservation { print("OBSERVATION:", observation) } } let inputObservation = VNDetectedObjectObservation(boundingBox: rect.boundingBox) let request: VNTrackingRequest = VNTrackObjectRequest(detectedObjectObservation: inputObservation, completionHandler: completion) request.trackingLevel = .accurate do { try videoProcessor.add(request, withProcessingOptions: [:]) try videoProcessor.analyze(with: CMTimeRange(start: .zero, duration: asset.duration)) } catch(let error) { print(error) }
A sample output I receive in the console during observation is;
OBSERVATION: <VNDetectedObjectObservation: 0x2827ee200> 032AB694-62E2-4674-B725-18EA2804A93F requestRevision=2 confidence=1.000000 timeRange={{0/90000 = 0.000}, {INVALID}} boundingBox=[0.333333, 0.138599, 0.162479, 0.207899]
I note that the observation reports something is invalid, alongside the fact that the confidence is always reported as 1.000000 and the bounding box coordinates never change. I'm unsure if this has to do with my lack of VNVideoProcessingOption setup or something else I am doing wrong.