Hello, so I was having this problem. I had this problem and I figured out a solution for me. I am using an Apple Silicon Mac so it probably only applies to this case. I had started launching Xcode with Rosetta initially because I was suffering build issues. However, after removing this setting i've found launch speed to be MUCH quicker, and I no longer suffer from the build issues in Xcode 13.
To disable rosetta, right click on the Xcode icon and choose "Get Info", then uncheck "Open using Rosetta" and relaunch Xcode and try again. Unlikely to be the solution for everyone but hopefully it's helpful to some.