AVPlayer realtime effects on iOS 13

I apply real time effects using CoreImage to video that is played using AVPlayer. The problem is when the player is paused, filters are not applied if you tweak filter parameters using slider.


let videoComposition = AVMutableVideoComposition(asset: asset, applyingCIFiltersWithHandler: {[weak self] request in

        // Clamp to avoid blurring transparent pixels at the image edges
        let source = request.sourceImage.clampedToExtent()
        let output:CIImage

        if let filteredOutput = self?.runFilters(source, filters: array)?.cropped(to: request.sourceImage.extent) {
             output = filteredOutput
         } else {
             output = source
         }


        // Provide the filter output to the composition
        request.finish(with: output, context: nil)
    })


I used this workaround to forcibly apply effects on paused player that worked only till iOS 12.4 unfortunately:


if self.moviePlayerView.player?.rate == 0.0 {
        self.moviePlayerView.player?.currentItem?.videoComposition = self.moviePlayerView.player?.currentItem?.videoComposition
    }

I am not sure what to do in iOS 13, how do I force apply effects on paused player?

Replies

I don't know the specific answer here, but I have an idea based some some similar code I did in the past. You need to get the pixel buffers from the source as they are being generated, and in most cases you will just immediately free them. However, when the user stops the video, you will then have the last pixelBuffer displayed. Convert that pixel buffer into a CG (or UI) Image, then get a CIImage from it using the same filters as you are using in the video. Display that CIImage over top of the video. Now, when the filters are modified, they will act on the display CIImage. When you restart the video, you may need to keep that image frontmost for a short period of time until the modified filters "kick in" to the video, then you can animate the over laid CIImage and the user will see the video using the modified filter settings.