Same here: using a magic mouse A1657 (magic mouse 2), mini mac from 2018 and a MacBook Air (2019). Your post made me realise problems started after updating the OS to Monterey. Now I am keeping it updated constantly, hoping the next version will have the problem solved. I am now running 12.5. Still the magic mouse ánd the magic keyboard are not
stable. The mouse indeed moves like a brick now and then, clicks are not handled half the time, selecting text has become a major challenge, moving windows across too and more... Started using the old magic mouse A1296 (magic mouse 1), for now this seems to work better. And as a spare I have a USB wired Dell ready.
But please Apple, do something really quick about it. It is very annoying for I earn my living working all day with these products that used to work perfect before updating the OS!!!