VNGeneratePersonSegmentationRequest produces mask at different resolution then source image

Trying to use VNGeneratePersonSegmentationRequest.. it seems to work but the output mask isn't at the same resolution as the source image.. so comping the result with the source produces a bad result.

Not the full code, but hopefully enough to see what I'm doing.

var imageRect = CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height)

let imageRef = image.cgImage(forProposedRect: &imageRect, context: nil, hints: nil)!

let request = VNGeneratePersonSegmentationRequest()
let handler = VNImageRequestHandler(cgImage: imageRef)

do {
                        
  try handler.perform([request])                 
  guard let result = request.results?.first else {
    return
  }

  //Is this the right way to do this? 
  let output = result.pixelBuffer           

  //This ciImage alpha mask is a different resolution than the source image 
  //So I don't know how to combine this with the source to cut out the foreground as they don't line up.. the res it's even the right aspect ratio.  
  let ciImage = CIImage(cvPixelBuffer: output)
   .....             
} 
Answered by DTS Engineer in 758707022

The resolution of the segmentation mask and the input image will rarely match, so you will need to scale the segmentation mask so that it matches the input image if you wish to apply it to the input image.

There is a code example of how to do this in this WWDC video: https://developer.apple.com/videos/play/wwdc2021/10040/?time=744 (This link is timestamped to the code example)

Accepted Answer

The resolution of the segmentation mask and the input image will rarely match, so you will need to scale the segmentation mask so that it matches the input image if you wish to apply it to the input image.

There is a code example of how to do this in this WWDC video: https://developer.apple.com/videos/play/wwdc2021/10040/?time=744 (This link is timestamped to the code example)

Ah hadn't seen that video, got it working. I was thrown because the mask output aspect ratio was different and didn't imagine I'd just have to scale it back.

Thanks for the help!

Daniel

VNGeneratePersonSegmentationRequest produces mask at different resolution then source image
 
 
Q