Running beta macOS builds on testers' computers

I've recently started testing our game on macOS, and was hoping to send out code-signed builds so my coworkers can test some of the Apple services we're using (i.e., iCloud). I've registered all of my coworkers' various Macs through the developer portal, created a Development certificate, and created a provisioning profile that includes all those devices, and the appropriate certificate.

I was having a problem, however: one of my coworkers was able to launch the game, while another wasn't – the app would crash, and the crash log would mention a code-signing error.

Ultimately, we were able to solve this by having this latter user install the provisioning profile, which I'd used to build the app, on their machine. Once that was done, they could launch the game without issue. (The other tester was also actively developing the game, and thus already had the profile installed on their system as well).

But this sort of goes against my understanding of the whole process of registering devices and creating certificates & profiles. As I understood it, so long as the user's device (its UUID) was registered on the dev portal, and included in the provisioning profile, this would be enough for them to run the game without any further work on their end.

Is there some reason why the tester would have to install the provisioning profile in order to launch the game?

I'll note that this is a game for Apple Arcade, in case that requires a bit of extra signing/provisioning.