The official Apple documentation (https://developer.apple.com/metal/tensorflow-plugin/) for tensorflow-metal (0.2) indicates that "Multi-GPU support" is not currently supported.
What does this mean? I could imagine that this means one of the following:
Option 1: Your M1 Mac has so many GPU cores (i.e. 32 cores in the case of M1-Max). Only 1 core is usable by tensorflow-metal currently.
Option 2: All GPU cores are used, but show up as a single core. Consequently, you can't and don't need to use Distributed Training (https://www.tensorflow.org/guide/distributed_training), one strategy to use multiple GPUs in TensorFlow (https://www.tensorflow.org/guide/gpu)
Can you clarify? Thanks.