Hello,
I'm new to the forums and iOS dev, but have a ML & software engineering background. I'm pretty intrigued by the promise of the GPUs and neural engine in Apple's mobile architectures, it introduces huge potential, and I want to explore it. Reading the developer docs and watching a couple of WWDC videos, it seems that yes you can convert models from Tensorflow to CoreML, but it's not clear to me whether they will use the "neural engine" (whatever that actually is) once converted. It seems that using Metal Performance Shaders is a way to (quasi-?)guarantee execution on the GPU, but it's not clear whether that's the same as the Neural Engine (it seems not). I'm assuming that models built with CreateML will be smart enough to use the Neural Engine, but I couldn't find that stated anywhere. Not to mention the other enhancements like quantization and half precision weights.
So the question is, to access the potential of the hardware, will I have to reimplement my model using the CreateML toolkit? How sophisticated is the Tensorflow converter when it comes to maximising the potential of the hardware?
I'm also curious about how to determine whether the code is executing on the hardware or not, but I suppose that's a different topic.
Edit to add - I've dug around a bit more and thanks to a couple of the Metal Computation videos from 2017 & 2018 WWDCs I now see it's possible to build our own CNNs & RNNs through the Metal Performance Shaders framework, and they will be executed on the GPU. However it's not yet clear to me how this relates to CoreML, or the Neural Engine, if at all.
Many thanks.
-Thom