It is now supported. The name of the Torch device is "mps" (for both older Intel macs with AMD GPUs and newer Macs with Apple Silicon). This is discussed in Apple docs and in this Pytorch thread. Nothing special is required, other than pip-installing the packages. (Note that the Apple docs are slightly out of date in that you no longer need to use nightly pytorch builds to have support for mps.) You can check this works with torch.rand(3, 3, device='mps'), which will fail if mps is not available.
As a tangent, for Tensorflow, you need to install the tensorflow-metal pypi dependency, discussed in Apple docs here. You can check this worked with tensorflow.config.list_physical_devices() (should include GPU:0 in the list)