I am experiencing this issue as well. I have a 2021 MacBook Pro (M1 Pro chip) and running on Monterey 12.2.1
This is really, really annoying considering I edit video for a living. This glitch adds unnecessary time to my edits because I have to constantly replay to make sure the audio is actually OK.
With the price point of these computers this should not be an happening, IMO.