For years (since I got my M1 MacBook Pro) I've experienced this weird performance degradation. I've tried posting about it to numerous channels, always without a reply. I figured I'd try here.
The symptom is: video playback (of visual frames, not audio; audio is unaffected) stutters if I just let the player play. I'll start a video playing, it will play normally, but if I leave the computer alone to play back, the frame rate will drop dramatically (one or two frames per second, even slower). If I wiggle the mouse (generating events), playback resumes normally. After a few seconds, it will stutter again.
It's as if the computer were throttling back on system clocks or something because there’s no user input.
As you can imagine, this is a frustrating user experience when trying to watch videos.
It doesn't matter if it's a YouTube video playing in Safari, or a WWDC video playing in the WWDC.app or Developer.app, or any other video in QuickTime Player.app.
And it doesn't always happen. I have my own video playback app that doesn't seem to suffer from this, even though it plays multiple videos simultaneously.
I'm on power adapter, and settings are generally set to maximum performance when on power adapter.
Any ideas?