Converting Keras InceptionV3 to Core ML Model

Hey guys,


My long-term plan is to retrain InceptionV3 on some custom image classes. I've been planning on using Keras for that, becuase it seems easy to use, I found some resources to help with my python script.


I'm running into some trouble, though.


As a first step, I'm trying to just convert the standard InceptionV3 trained off of the ImageNet set. I run some test images against them, and get results I'd expect. However, once I convert the model, I get different results. Similar, but different.


I tried running it against Apple's InceptionV3 that they provide, and they provide correct results, similar to the unconverted Keras model.


Has anyone done this? I feel like I'm missing some sort of step to prepare the input, like maybe I need to add another input layer to Keras, or something.


For ease of use, I've posted my source code to github:

https://github.com/vml-ffleschner/coremltools-keras-inception-test


I'll keep hacking on this and post back if I figure something out, but hopefully someone can easily see what I'm doing wrong!

Accepted Reply

BTW, I've figured this out.


It was a matter of using the correct parameters in the convert call:


image_scale=2./255

red_bias=-1

green_bias=-1

blue_bias=-1


My code example has been updated to work.


Also, the way I figured this out was to print out the contents of MLModel.get_spec() for both of the models into a file (each was ~450MB!!!), and then diff them.

Replies

BTW, I've figured this out.


It was a matter of using the correct parameters in the convert call:


image_scale=2./255

red_bias=-1

green_bias=-1

blue_bias=-1


My code example has been updated to work.


Also, the way I figured this out was to print out the contents of MLModel.get_spec() for both of the models into a file (each was ~450MB!!!), and then diff them.

@singerfrank, Thanks for sharing. I am an 'expert beginner' with deep learning but am brand new to ios development. I, also have retrained and fine-tuned the keras inceptionV3 model. I am confused on how the apple coreml model accepts an RGB image. Where do they do their pre-processing? Did I do something wrong in my conversion if the Xcode project is requesting the array of image data (MultiArray<Double,3>) instead of an RGB object sized to 299,299?


https://drive.google.com/file/d/0Bzg5-HDFz60RSHY5eTBiLVp6Z0k/view?usp=sharing


Update: I found a post that indicated it might be a versioning issue between all my python packages. I am going to make sure that I have all the proper dependencies installed and try again. Will update with results.


Update: I had Tensorflow==1.1.0 & Keras==2.0.4 & Numpy==1.13.0 & Protobuf==3.3.0, all of which should match the requirements of coremltools. It failed because of a keras incompatability which I resolved by removing the Optimizer Weights from the keras model which prevents me from continuing training the model but that is not a problem. I converted this model to the coreml model format using your same params. It is still resulting in model with a required input of MultiArray<Double,3>. It would be optimal if I could solve this on the conversion so that I can simply pass it the CVPixelBuffer. But, I am ultimately confused about the pre-processing in coreML and how to feed the image to the model properly when in python I have to use the keras.applications.inception_v3.preprocess_input(). Did you retrain your model without the pre-processing function?


Any help would be greatly appreciated!

I struggled with the same problem, but I figured out yesterday that you need to provide both the `input_names` and `image_input_names` arguments when converting the model in order for the input to be interpreted as an image. E.g.

coremltools.converters.keras.convert('inception_model.h5',
    input_names='image', image_input_names='image', class_labels='labels.txt',
    red_bias=-1.0, blue_bias=-1.0, green_bias=-1.0, image_scale=2./255)