The culprits were actually different ones and not in the code I posted. ) The kernel_task: The following code contains the memory leak: let croppedImage = frame.image.cropped(to: CGRect( 		x: Int(Int(videoSize.width) / 2 - (self.RECOGNITION_WIDTH / 2)), 		y: Int(self.videoSize.height - CGFloat(self.RECOGNITION_HEIGHT)), 		width: self.RECOGNITION_WIDTH, 		height: self.RECOGNITION_HEIGHT ) Apparently CIImage.cropped(to:) leads to the original CIImage not being cleaned up. According to my Google search I should have called CGImageRelease(:) but that is not contained in Swift 5.1 since Core Foundation Objects are now memory managed. (At least that is the Xcode error message.) So I convert the image to a CGImage, crop that one and turn it into a CIImage. I know this is not efficient, but for me that's fine. So here is the new code: import Foundation import CoreImage extension CIContext { 		func cropWithoutMemoryLeak(_ image: CIImage, to rect: CGRect) -> CIImage 		{ 				let newImage = self.createCGImage(image, from: rect) 				return CIImage(cgImage: newImage!) 		} } ) The VTDecoderXPCService: I was able to reliably remove the memory leak by commenting out this line, which obviously defeats the point of creating a new video: self.frameInput!.append(newSampleBuffer) After trying some more stuff I removed some of my debug print statements which I had throughout my code. And all the sudden the leak was gone...? 🤷‍♀️ So I don't really know what the problem was. Removing the debug lines was most certainly a correlation but no causation. I guess it triggered a recompile of previously cached files but admittedly I have no idea, what the problem was.