Render video in metal.

So I wanted to render video from .mov file in MTKView.

Short algorithm:

  1. Read CMSampleBuffer from .mov using AVAssetReader and AVAssetReaderTrackOutput.

  2. Convert CMSampleBuffer from step 1 to MTLTexture and pass it to renderer

I did those 2 steps and got the picture with twitches and I don’t know why.

Link to .mov https://www.dropbox.com/s/gmzxd8j94pjhc1q/2.MOV?dl=0.

Link to result https://www.dropbox.com/s/exgf1tk7oqvon25/result.mov?dl=0.

The code.

Instead of a Timer in VideoReader.swift, try using CADisplayLink with a selector that points to this method

 @objc private func readBuffer(_ sender: CADisplayLink) {

        let nextVSync = sender.timestamp + sender.duration

        let currentTime = playerItemVideoOutput.itemTime(forHostTime: nextVSync)

        if playerItemVideoOutput.hasNewPixelBuffer(forItemTime: currentTime),

           let pixelBuffer = playerItemVideoOutput.copyPixelBuffer(forItemTime: currentTime, itemTimeForDisplay: nil),

           let f = getClosestFrame(time: currentTime.seconds) {

            // Code 

        }

    }
Render video in metal.
 
 
Q