How to create a immersive panorama like the environments?

I would like to create a immersive panorama like the environments where the user can look around 360°, yet interactive, i.e. the user should be able to interact with entities placed on that panorama.

My current approach is to create a sphere around the user and invert the normals, so the texture is placed inwards, towards the user. This works, but open SwiftUI windows show pretty weird behaviors, as described here:

https://developer.apple.com/forums/thread/749956

Windows don't show their handles anymore, and the glass effects do not recognize my sphere but show the world "outside" of it. This is not the case for Apple's environments.

Is there a better way to create a fully immersive sphere around the user?

How to create a immersive panorama like the environments?
 
 
Q