I have the same issue (newest 2021 mbp with M1pro).
But had it also in previous mbp (2016) - thus I believe it is software thing. In the past it was working as expected. Started to happening after some OS upgrade (already thought it is only my issue, but it is reproducible on few 2021 mbpros with M1pro and OS 12.1).
It is annoying, reproducible with COMMAND or SHIFT.