coremltools does not support Conv3D?

I'm working on fine tuning a model which needs to be ported to CoreML for a project. I find that reshaping the data into a 3D array and applying 3D convolutional layers yields massively better results for this particular problem.


Unfortuantely, when I then try to export the model using coremltools 0.7 with Keras 2.1.1, I get the following error:


  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/coremltools/converters/keras/_keras_converter.py", line 722, in convert
    custom_conversion_functions=custom_conversion_functions)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/coremltools/converters/keras/_keras_converter.py", line 527, in convertToSpec
    custom_conversion_functions=custom_conversion_functions)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/coremltools/converters/keras/_keras2_converter.py", line 169, in _convert
    _check_unsupported_layers(model, add_custom_layers)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/coremltools/converters/keras/_keras2_converter.py", line 88, in _check_unsupported_layers
    raise ValueError("Keras layer '%s' not supported. " % str(type(layer)))
ValueError: Keras layer '<class 'keras.layers.convolutional.Conv3D'>' not supported.


I was under the impression that coremltools supports export of convolutional layers. I had no problem before I reshaped the data, when I was simply using Conv2D on the same data. Is there some way to get the model to export as is, without having to give up the gains from using the 3D convolutions?

Up vote post of jmp_nyc Down vote post of jmp_nyc
1.1k views
Posted by
jmp_nyc
Reply
Add a Comment

Replies

If you look at NeuralNetwork.proto in the coremltools source code, you'll see that kernelSize in ConvolutionLayerParams must be length 2. So it looks like it only supports 2D convolutions at the moment.


The technologies that are backing Core ML on iOS, i.e. BNNS and Metal, also only support 2D convolutions.


If you want to use a 3D convolution, I'm afraid you're going to have to write a custom Core ML layer.

Posted by
kerfuffle
Up vote reply of kerfuffle Down vote reply of kerfuffle
Add a Comment

Is the a limit to the number of channels it can handle in a 2D matrix? Perhaps I could rotate my data to have a dimension over which I want to have each convolutional kernel look at the entire range treated as the channel, to get the same effect without having to write my own Core ML Layer.

Posted by
jmp_nyc
Up vote reply of jmp_nyc Down vote reply of jmp_nyc
Add a Comment