Incorrect predictions with CoreML

Adding few more details if those may help. In my Python predictor script, I am doing the following preprocessing to the input before it's passed to the model for predictions:

img = image.load_img(img_path, target_size=(299, 299))

x = image.img_to_array(img)

x = np.expand_dims(x, axis=0)

x = preprocess_input(x)

---------------------------------

iOS code (Simplified for understanding):

ciImage = CIImage(image: selected_image)

let request = VNCoreMLRequest(model: model) { [weak self] request, error in

guard let results = request.results as? [VNClassificationObservation],

let topResult = results.first else {

fatalError("unexpected result type from VNCoreMLRequest")

}

let handler = VNImageRequestHandler(ciImage: image)

DispatchQueue.global(qos: .userInteractive).async {

do {

try handler.perform([request])

} catch {

print(error)

}

}

As depicted above, the iOS code doesn't have any preprocessing applied before it's passed to VNCoreMLRequest. Does Vision(VNImageRequestHandler) take care of the same?

Thanks,

Rishi

You do preprocessing in Python on the input. But your Core ML model doesn't. So the Python model gets different inputs than the Core ML model, and naturally the results will be different then too.

When you create the Core ML model you need to tell it what preprocessing to do (scale the pixels, subtract RGB means, flip RGB to BGR, etc).

Apple should make this an FAQ on their machine-learning page because it's the number 1 thing that trips people up with Core ML.