I had a similar experience, it stopped working, no amount of pairing / rebooting would make it work. 2 Days later it suddenly started working without even re-pairing it.
Sadly 2 days later it stopped working and I'm stuck again.
XCode 13.1, tvOS 15.1.