Concatenating models

Hi,


I'm trying to create a mlmodel using the python package corermltools. I'm trying to concatenate 2 models like :

image_model = createImageModel()

lang_model = createLanguageModel()

model = concatenate([image_model.output, lang_model.output])

I can create the model and training it properly but when i tried to convert I get this:


Traceback (most recent call last):

File "/Users/user/Pers/captioning_images/sources/main.py", line 100, in <module>

convertToCoreML(model)

File "/Users/user/Pers/captioning_images/sources/main.py", line 19, in convertToCoreML

coreml_model = convert(model)

File "/Users/user/anaconda/envs/captioning-image/lib/python2.7/site-packages/coremltools/converters/keras/_keras_converter.py", line 504, in convert

40 : concatenate_1, <keras.layers.merge.Concatenate object at 0x11bb12450>

predicted_probabilities_output = predicted_probabilities_output)

File "/Users/user/anaconda/envs/captioning-image/lib/python2.7/site-packages/coremltools/converters/keras/_keras2_converter.py", line 278, in _convert

converter_func(builder, layer, input_names, output_names, keras_layer)

File "/Users/user/anaconda/envs/captioning-image/lib/python2.7/site-packages/coremltools/converters/keras/_layers2.py", line 511, in convert_merge

mode = _get_elementwise_name_from_keras_layer(keras_layer)

File "/Users/user/anaconda/envs/captioning-image/lib/python2.7/site-packages/coremltools/converters/keras/_layers2.py", line 79, in _get_elementwise_name_from_keras_layer

raise ValueError('Only channel and sequence concatenation are supported.')

ValueError: Only channel and sequence concatenation are supported.


Thanks

Up vote post of joseluis.alcala Down vote post of joseluis.alcala
1.8k views
Posted by
joseluis.alcala
Reply
Add a Comment

Replies

Hello @joseluis.alcala


I'm trying to do something similar (concatenate two models). In my case one is a caffe net and the other a skilearn classifier.


I can't find any documentation about this concatenation. I was experimenting using coremltools.models.pipeline but no luck. Could you point me to some documentation about this ?


Best regards

Posted by
jfuentes
Post not yet marked as solved Up vote reply of jfuentes Down vote reply of jfuentes
Add a Comment

Hi, thank you for using coremltools!


The code in your post seem to have issues. In Keras, concatenate layer should return a Tensor rather than model. Running your code I get the following error:

AttributeError: 'Tensor' object has no attribute 'layers'


Have you tried changing your code into this?

# x1 and x2 are Input layers from model1 and model2
  z = concatenate([model1.output, model2.output])
  model = Model([x1,x2], z)


Apart from that, CoreML only supports concatenation on feature channels and sequences. In Keras, for example, if Tensor x1 and x2 both have shape (Batch, Sequence, Channels), then only concatenating at axis=1 (sequence) or axis=2 (channels) is supported.


Thanks!

Posted by
allenlin
Up vote reply of allenlin Down vote reply of allenlin
Add a Comment