Style transfer models using much more memory on iPhone Xs

Hi Frank, please file a bug report and update your post to mention the bug number. Thanks.

Hey, thanks for the fast response!

I filed a radar (44821525) and also attached a model for reproducability.

Here are some findings and a work-around I found:

From what I could see in Instruments I conclude that the CoreML runtime is pre-allocating all buffers (hence the many IOSurfaces) required to execute the neural network when initializing the model (using the method Espresso::ANERuntimeEngine::blob_container::force_allocate()).

Interestingly this only happens for models with a relatively high input size (1792 x 1792) and not for smaller ones (1024 x 1024).

Since this is only happening on the Xs I assumed it has something to do with the A12’s Neural Engine. So I configured the model to use CPU and GPU only as compute unit (MLComputeUnitsCPUAndGPU instead of MLComputeUnitsAll) and that did the trick—no more pre-allocated buffers.

So I’m using this as a work-around for now.

Hi CoreOSTechE,

Are there any updates on this issue? I just tested on iOS 12.1 and the issue remains the same.

Thanks!

I have also noticed that models on the XS require more memory than the same model on older devices, and that this goes away when not using the Neural Engine. The difference wasn't as extreme as what you found, but still a noticeable different nonetheless.