I just upgraded to Big Sur 11.1 on WED 16-dec-2020 .. and was facing this issue as well. When I launched iMessage and FaceTime from the dock, they just wouldn't start and had to be killed via "Force Quit Applications". I even tried to launch iMessage via Terminal (as mentioned by someone else on this thread), but that also did not work.
Then I fixed it : I clicked on : (apple) > System Preferences > Internet Accounts > and then click on each account listed there.
Turns out, when my MBA rebooted after the upgrade, it also logged out of my Google account. For some strange reason, that was impacting the launching of iMessage and FaceTime. After I re-logged into my Google account (via 2FA in my case), then it all started working fine again.