I was getting an error installing it via App Store. However, this worked for me. After I tried (and failed) to install it from App Store, I found that I have a new Application:
/Applications/Install macOS Big Sur
I right-clicked that, clicked "Show Package Contents", and navigated to Contents -> MacOS and I double-clicked InstallAssistant.
I was able to install it from there