Interestingly, it happens when I open Xcode while my iPhone is connected to the machine.
Even though it's not the active build target, having it connected means Xcode will freeze soon after opening any project.
So the fix was easy: Disconnect iDevice
Open Xcode
Wait a bit
Connect device again