I was having a similar issue.
You can verify if this is your issue by pulling up Activity Monitor -> CPU -> Xcode and see if the "Kind" is Apple or Intel or follow the directions here
If it's Intel and you recently downloaded the latest version of Xcode on an M1 Mac you might be confused. This can be the case if you're launching using Rosetta (might have been something you set awhile ago and it persists when you replace the app downloaded from the web, can't comment on if the App Store version persists this or not).
To fix it
Go to the directory containing the Xcode app, right click, get info and uncheck the "Open using Rosetta"