How can I pinch to open a menu in VisionOS simulator?

Hello, Here is my code below.

struct ImmersiveView: View {
    @Environment(\.presentationMode) var presentationMode
    @GestureState private var scale: CGFloat = 1.0
    @State private var showMenu = false
    
    var body: some View {
        ZStack {
            RealityView() { content in
                // Get the video URL
                //Create Entity for the video
                let videoEntity = Entity()

                //Search for video in paths
                guard let url = Bundle.main.url(forResource: "harmandir-sahib-sarovar", withExtension: "mp4") else { fatalError("Video was not found!") }
                
                //create a simple AVPlayer
                let asset = AVURLAsset(url: url)
                let playerItem = AVPlayerItem(asset: asset)
                let player = AVPlayer()

                //create a videoMaterial
                let material = VideoMaterial(avPlayer: player)

                //Made a Sphere with the videoEntity and asign the videoMaterial to it
                videoEntity.components.set(ModelComponent(mesh: .generateSphere(radius: 1E3), materials: [material]))

                //adjust the properties of the videoEntity(Sphere) if needed
                videoEntity.scale = .init(x: 1, y: 1, z: -1)
                videoEntity.transform.translation += SIMD3<Float>(0.0, 10.0, 0.0)

                let angle = Angle.degrees(90)
                let rotation = simd_quatf(angle: Float(angle.radians), axis: .init(x: 0, y: 0, z: 0))

                videoEntity.transform.rotation = rotation

                //add VideoEntity to realityView
                content.add(videoEntity)
                
                //start the VideoPlayer
                player.replaceCurrentItem(with: playerItem)
                //set the actionAtItemEnd property to .none
                player.actionAtItemEnd = .none
                
                //subscribe to the notification and seek back to start
                NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: player.currentItem, queue: .main) { _ in
                    player.seek(to: CMTime.zero)
                    player.play()
                }
                
                player.play()
            }
            Color.clear // An invisible view that covers the whole screen
            .gesture(MagnificationGesture().updating($scale) { value, state, transaction in
                    state = value
            }.onEnded { value in
                    showMenu.toggle()
            })
        }
        .overlay(
            Group {
                if showMenu {
                    VideoView(dismissAction: {
                        presentationMode.wrappedValue.dismiss()
                    })
                }
            }
        )
    }

}

struct VideoView: View {
    let dismissAction: () -> Void
    
    var body: some View {
        VStack {
            Button("Back") {
                dismissAction()
            }
        }
        .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .bottom)
        .background(Color.white)
    }
}

I am working on a way to pinch anywhere on the screen in the VisionOS simulator to open up a menu. Unfortunately the coding above isn't working. Can you please help me out. Thank you.

SwiftUI views in visionOS can be presented as windows via the openWindow enviroment action and as RealityView attachments. Your VideoView is presented as neither. An ImmersiveSpace doesn't know where to put a SwiftUI view on its own in the unbounded full space.

Note that if you go for an attachment, be wary of the recent API changes which seem to be currently undocumented.

                // Get the video URL
                //Create Entity for the video
                let videoEntity = Entity()

                //Search for video in paths
                guard let url = Bundle.main.url(forResource: "harmandir-sahib-sarovar", withExtension: "mp4") else { fatalError("Video was not found!") }
                
                //create a simple AVPlayer
                let asset = AVURLAsset(url: url)
                let playerItem = AVPlayerItem(asset: asset)
                let player = AVPlayer()

                //create a videoMaterial
                let material = VideoMaterial(avPlayer: player)

                //Made a Sphere with the videoEntity and asign the videoMaterial to it
                videoEntity.components.set(ModelComponent(mesh: .generateSphere(radius: 1E3), materials: [material]))

                //adjust the properties of the videoEntity(Sphere) if needed
                videoEntity.scale = .init(x: 1, y: 1, z: -1)
                videoEntity.transform.translation += SIMD3<Float>(0.0, 10.0, 0.0)

                let angle = Angle.degrees(90)
                let rotation = simd_quatf(angle: Float(angle.radians), axis: .init(x: 0, y: 0, z: 0))

                videoEntity.transform.rotation = rotation

                //add VideoEntity to realityView
                content.add(videoEntity)
                
                //start the VideoPlayer
                player.replaceCurrentItem(with: playerItem)
                //set the actionAtItemEnd property to .none
                player.actionAtItemEnd = .none
                
                //subscribe to the notification and seek back to start
                NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: player.currentItem, queue: .main) { _ in
                    player.seek(to: CMTime.zero)
                    player.play()
                }
                
                player.play()
            }
            .contentShape(Rectangle())
            .onTapGesture {
                Task {
                    openWindow(id: "Video")
                }
            }
        }
    }

}

I did create a WindowGroup of that id Video to open a window with a back button. But unfortunately I have tried loads of methods especially with the .onTapGesture and when I try to tap anywhere on the screen in the VisionOS simulator but it still would not open the window. What I really want is to enable tap gestures in the simulator so that it can open the window by a single touch anywhere on the 360 panorama video (reality view).

I do not know what I have been doing wrong. I really need some help. Thank you.

You should give SpatialTapGesture a try. It can be targeted to any or to specific entities with targetedToEntity() or targetedToAnyEntity().

.gesture(
	SpatialTapGesture()
		.targetedToEntity(self.videoEntity)
		.onEnded { _ in
			Task {
				openWindow(id: "Video")
			}
		}
)

Unfortunately, when I tried to use this gesture method at the end of RealityView() {... } it would not work. Even when I tried to do targetedtoEntity() it still would not open the window via tapping on the simulator when in the immersive view which is the 360 panorama video. Can you please provide me with some more help? Thank you.

It's hard to tell what's going wrong without seeing your actual code. Have you made sure that your tap gesture closure is actually being called? Have you made sure your openWindow statement actually opens the specified window?

This is my main app coding with all the window groups. I am trying to ensure that the Video window opens when I tap anywhere on the screen in the immersive view space named "Immersive Space".

    var body: some Scene {
        WindowGroup(id: "Begin") {
            MainMenuView()
        }
        
        WindowGroup(id: "Navigate") {
            MapView()
        }
        
        WindowGroup(id: "Video") {
            VideoView()
        }

        ImmersiveSpace(id: "ImmersiveSpace") {
            ImmersiveView()
        }.immersionStyle(selection: .constant(.full), in: .full)
    }
}

Here is my VideoView coding which displays the back button. This is the view I want to open in the window of the visionOS simulator when I tap anywhere on the screen.

    @Environment(\.dismiss) private var dismiss
    
    var body: some View {
        VStack {
            Button("Back") {
                dismiss()
            }
        }
        .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .bottom)
        .background(Color.white)
    }
}

Is this coding knowledge enough for you to help me? Thank you.

I don’t see code in your recent post that invokes the open window command.

However, without an entity to receive the tap, it may not be possible to do what you want.

An empty Immersive View doesn’t have anything to receive the tap. You may need to use the scene reconstruction provider to generate meshes of the world so those meshes can receive the taps. This is not available in the simulator at the moment.

How can I pinch to open a menu in VisionOS simulator?
 
 
Q