Same issue here.
Deleting the contents of ~/Library/Developer/Xcode/iOS DeviceSupport (also logs and caches) didn't solve the problem for me.
With this configuration: Big Sur 11.3
Xcode 14.5
Simulator 14.5
my app start takes more than 30 seconds when running from xcode. Normal start of installed app takes 2 seconds.
Xcode says "launching app", "attaching to app", and then "running app", at the running app stage I get a ~30 second pause. debugserver at 100% activity at that time.
Unticking "Debug executable" makes start instantaneous but makes debugging impossible, it's not a solution.
On the real device (running iOS 14.4) start with attached debugger takes around 6 seconds.