Happening for me on 11.6 (M1 MackBook Air). Might be associated with having an iPad app running (Overcast). I find if I use control centre to switch from Bluetooth to built-in for a moment and then back again the problem goes away. However as soon as something changes, e.g. next podcast in Overcast, you have to flick to speakers and back again once more.