Follow-up on my situation. I was able to re-install MacOS from booting into Recovery mode. I had to delete local time machine snapshots first for it to think it had enough space (even though I had 40+GB free and clear (not tied up as purgeable). But installing it that way seems to have started over from scratch on the install and avoided the infinite reboot issue I was having when trying it through System Prefs. So far things seem back to normal (as normal as a beta can be...).

When I tried to do the update from beta 3 to beta 4, it went into an infinite reboot loop. Progress bar might go 10% then reboot, and repeat. Over and over. I can use the Option key to select the startup disk and boot up, but the machine does not act 'normal' anymore (touch security doesn't work for instance). If I launch Disk Utility and look at the container for my volumes, I see volumes I shouldn't be seeing (preboot, not mounted, etc.). I can also launch into recovery mode and disk utility comes up fine. If I look at my Time Machine backups, the backups associated with 'Macintosh HD' are 10.15. If I look at the backups for 'Macintosh HD - Data', they are 11. The machine was on 10.15.6 before installing Big Sur Beta 1. If I try to restore from one of the version 11 backups, it tells me I need to reinstall MacOS then restore. If I go to reinstall, the only option it gives me is 10.15 to reinstall. Something definitely went wonky in the update from 3 to 4. Haven't decided what my next step will be, but documenting here for woes on the 3 to 4 update. Seems like it failed, and left temporary volumes on my drive. Prefs keeps telling me that Beta 4 is available, but any time I try to install via Prefs, it goes back to it's infinite reboot loop.