Instead of restarting your Mac just turn the mouse on and off, its a much quicker way to get past this. I have MB Air M1 and its been doing it for awhile now.
ja-rad
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution
From
Bellingham, WA