AVFoundation: scaling a video change colors

I'm creating a video composition by scaling a single video using AVFoundation and an AVMutableComposition. When I apply any scaling other than 1 (either bigger or smaller), I get a color shift on the end result.

I have this issue regardless of the video used as source or the preset/format used as destination (MOV/MP4, ProRes/H24...).

Here is the relevant code, widely inspirited from another StackOverflow question (https://stackoverflow.com/questions/35402633/swift-video-resizer-avasset)

func resizer(inputURL: URL , completion: @escaping (URL?) -> Void ){
     let videoAsset = AVAsset(url: inputURL) as AVAsset
     let clipVideoTrack = videoAsset.tracks(withMediaType: AVMediaTypeVideo).first! as AVAssetTrack
  
     let composition = AVMutableComposition()
     composition.addMutableTrack(withMediaType: AVMediaTypeVideo, preferredTrackID: CMPersistentTrackID())
  
     let videoComposition = AVMutableVideoComposition()
     videoComposition.renderSize = clipVideoTrack.naturalSize
     videoComposition.frameDuration = CMTimeMake(1, 30)
  
     let instruction = AVMutableVideoCompositionInstruction()
     instruction.timeRange = CMTimeRange(start: kCMTimeZero, duration: clipVideoTrack.asset!.duration)
  
     let transformer : AVMutableVideoCompositionLayerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: clipVideoTrack)


     // If anything that 1 on both, colors are shifted!
     let transform = CGAffineTransform(scaleX: 1.001, y: 1.001)
     transformer.setTransform(transform, at: kCMTimeZero)
     instruction.layerInstructions = [transformer]
     videoComposition.instructions = [instruction]
  
     // Export part, left for facility  
     let exporter = AVAssetExportSession(asset: videoAsset, presetName: AVAssetExportPresetHighestQuality)!
     exporter.videoComposition = videoComposition
     let outputURL = URL(fileURLWithPath: "/path/to/destination/video.mov")
     exporter.outputURL = outputURL
     exporter.outputFileType = AVFileTypeQuickTimeMovie
  
     exporter.exportAsynchronously(completionHandler: { () -> Void in
          completion(outputURL)
     })
}

This question has also been asked on StackOverflow: https://stackoverflow.com/questions/48888394/avfoundation-scaling-a-video-change-colors