With 11.4 RC my app is starting a lot faster than 11.3.1 but still a little slower than it was a month ago. And once started it's responsive immediately, unlike before where I had to wait another 20 seconds or so.
Not perfect but at least I can leave Debug Executable checked without needing to find a way to kill time.
This is on a 13" 2017 MacBook Pro.