I have the similar issue with Macbook Pro Apple Silicon running on Big Sur.
I've updated to Big Sur 11.2.3 and it did not really help.
The issue is continuously losing connect to external monitor or tv. I tried different cables: USB-C HDMI, USB-C HUB HDMI and different monitors Philips Brilliance 328P, Philips Brilliance 241P and Samsung TV.
The same issue for all the cables and monitors.
I am really sick of it, cannot use an external display for fastest laptop ever 🤦♂️
Should it help if I use USB-C DisplayPort?