I had exactly the same problem and I decided to delete Xcode and all simulators that I had. After 5 years of continuous every (work) day Xcode usage it had a lot of garbage there. Then I installed Xcode again and things Simulator is now working properly.
Simulators are located under /Users/user/Library/Developer/CoreSimulator/Devices
Btw. when I was emptying Bin it had more than 1 million files to delete and I assume that was the problem why Simulator was so slow.