How much memory is available for CNN

I think MPSImage just should be as a reslut for your CNN before render pass. MPSTemporaryImage is different from MPSImage which located memory if you read document of it. The memory with heap concept for temporary image with readcount to avoid from frequent allocation and destroy .

Maybe you should check your other kernels for cnn calculation ,like svd,mean or other globel calculation in your kernel which cost much time and resources ,and details refer to https://developer.apple.com/metal/limits/

Good luck