@fpappledeveloper - I was able to resolve the issue and you are correct. When you open the web inspector it immediately resets logins (cookies?).
I resolved the issue. Here's what I tried, sorry I didn't pin down exactly what fixed it:
Resetting all experiments: Develop -> Experimental Features -> Reset all to defaults
Re-enabling Safari -> Preferences -> Privacy -> enable "Prevent cross site tracking" + enable "Allow privacy-preserving measurements of ad effectiveness" (I suspect this is the issue but not sure)
Disabling all content blockers on various websites (i don't think this does anything)
Disabling Syncing Platform Authenticator
Updating macOS from 12.4 to 12.5