The CoreML MultiArray Float16 input is not supported for running on the NPU, and this issue only occurs on the iPhone 11.

Question

laumt OP

Created Sep ’24

Replies 3

Boosts 0

Participants 2

Xcode Version: Version 15.2 (15C500b)

com.github.apple.coremltools.source: torch==1.12.1 com.github.apple.coremltools.version: 7.2

Compute: Mixed (Float16, Int32)

Storage: Float16

The input to the mlpackage is MultiArray (Float16 1 × 1 × 544 × 960)

The flexibility is: 1 × 1 × 544 × 960 | 1 × 1 × 384 × 640 | 1 × 1 × 736 × 1280 | 1 × 1 × 1088 × 1920

I tested this on iPhone XR, iPhone 11, iPhone 12, iPhone 13, and iPhone 14. On all devices except the iPhone 11, the model runs correctly on the NPU. However, on the iPhone 11, the model runs on the CPU instead.

Here is the CoreMLTools conversion code I used:

mlmodel = ct.convert(trace,
                     inputs=[ct.TensorType(shape=input_shape, name="input", dtype=np.float16)],
                     outputs=[ct.TensorType(name="output", dtype=np.float16, shape=output_shape)],
                     convert_to='mlprogram',
                     minimum_deployment_target=ct.target.iOS16
                     )

Boost

Answer 1

laumt OP

Sep ’24

The test results of the same model on iPhone 11 and iPhone XR. iPhone XR runs correctly on ANE, while iPhone 11 runs on CPU

0

Answer 2

laumt OP

Sep ’24

The test results show that the issue occurs on all devices equipped with the Apple A13 Bionic chip.

0

Answer 3

Frameworks Engineer OP

Apple

Sep ’24

The compute unit assignment depends on:

Hardware (SoC),
OS software version, and
Operations in the model.

Core ML framework has a cost model to determine the optimal compute unit for each operation. It could be the case that the cost model thought that CPU was more optimal than Neural Engine for the given environment.

It's hard to say more without looking at the actual model. Could you file a feedback assistant with the model attached? Please also use the latest iOS 18.

0