There was an issue in the past on coremltools that was closed saying this is the appropriate forum for discussing how to get CoreML models to run on the Neural Engine: https://github.com/apple/coremltools/issues/337. I have a tensorflow model where the vast majority of layers can run on the GPU or Neural Engine. Conceptually, I don't see why all of it can't use the Neural Engine. I see that there are a couple layers associated with the GRU cannot run on the Neural Engine like get_shape (even though all of the shapes are known). Coremltools spit out the converted model, so I don't have much insight to why dynamic dimension layers are used instead of static dimensions. Is there any way to have some of the model inferenced on the GPU/NE or have coremltools guarantee that a generated model runs on the NE? I converted the model with coreml_model = ct.convert(probability_model, convert_to='mlprogram', compute_precision=ct.precision.FLOAT16, compute_units=ct.ComputeUnit.ALL) where ct is coremltools and probability_model is a Tensorflow 2 keras model that has a GRU in it. Some of the similar models I tried without the GRU run 20-30x faster on the NE. Here is an example performance report screenshot: One thing I notice that doesn't seem to match my expectations with coremltools is the storage and compute types differ. I don't know why because I exported from coremltools with float16 compute precision.

The new installation instructions for tensorflow at https://developer.apple.com/metal/tensorflow-plugin/ don't work. The Conda script name is wrong. The old instructions would share a download link to a script with the same names as in the example shell instructions The conda environment now installs 3.9 instead of 3.10, which is behavior that was changed without documentation. Tensorflow cannot be imported because it is compiled against a different version of numpy than is installed in the downloadable conda environment. A manual, python -m pip install --upgrade numpy is required to run the test script. The test script throws several warnings but works after the numpy upgrade. UserWarning: "sparse_categorical_crossentropy received from_logits=True, but the output argument was produced by a Softmax activation and thus does not represent logits. Was this intended? Further issues may be related to the M1 Ultra. Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. Failed to get CPU frequency: 0 Hz My experiment was totally isolated while unboxing a new Mac Studio. I installed Xcode command line tools, then went to the developer documentation linked before, and bam.