Can't convert any style transfer models with Coremltools

I am trying to find a feed-forward style transfer model that can be converted with coremltools. So far, everything I have tried fails.


I found a project that uses the trained model as a Keras model 1.1 model (most seem to use Keras 2) and attempting to covert it I get the error:


>>> coreml_model = coremltools.converters.keras.convert("water-lilies.h5")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/coremltools/converters/keras/_keras_converter.py", line 273, in convert
    model = _keras.models.load_model(model)
  File "/Volumes/SSD/Users/jeshua/tensorflow/lib/python2.7/site-packages/keras/models.py", line 142, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/Volumes/SSD/Users/jeshua/tensorflow/lib/python2.7/site-packages/keras/models.py", line 193, in model_from_config
    return layer_from_config(config, custom_objects=custom_objects)
  File "/Volumes/SSD/Users/jeshua/tensorflow/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 40, in layer_from_config
    custom_objects=custom_objects)
  File "/Volumes/SSD/Users/jeshua/tensorflow/lib/python2.7/site-packages/keras/engine/topology.py", line 2582, in from_config
    process_layer(layer_data)
  File "/Volumes/SSD/Users/jeshua/tensorflow/lib/python2.7/site-packages/keras/engine/topology.py", line 2560, in process_layer
    custom_objects=custom_objects)
  File "/Volumes/SSD/Users/jeshua/tensorflow/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 35, in layer_from_config
    instantiate=False)
  File "/Volumes/SSD/Users/jeshua/tensorflow/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 125, in get_from_module
    str(identifier))
ValueError: Invalid layer: ReflectionPadding2


The water-lillies.h5 model is here. I am guessing it fails because ReflectionPadding2 is a custom layer type. All the other Keras projects I can find either result in a similar error or are written for Keras 2.


I have tried to convert jcjohnson's trained Torch model to a Caffe model with Facebook's torch2caffe utility (based on fblualib) and I get the error:

unknown Torch class <nn.ShaveImage>


Apparently ShaveImage is not a standard Torch class, so it is not possible to convert.


In the WWDC video, Apple specifically mentions Style Transfer as a use of Core ML. Does anyone know of any style transfer models that will work with Core ML?


Any plans of supporting Torch models with Coremltools?

Replies

Here is a resource for finding CoreML models, that includes a few style transfer examples. http://mlmodelzoo.com/

I just released converter from torch7 to CoreML with an example how to convert Justin Johnson's feedforward models: https://github.com/prisma-ai/torch2coreml/