Znear For RealityView to prevent Entity from Hiding an attachment SwiftUI Button in VisionOS

I have an app which have an Immersive Space view and it needs the user to have a button in the bottom which have a fixed place in front of the user head like a dashboard in game or so but when the user get too close to any3d object in the view it could cover the button and make it inaccessible and it mainly would prevent the app for being approved like that in appstoreconnect I was working before on SceneKit and there was something like camera view Znear and Zfar which decide when to hide the 3d model if it comes too close or gets too far and I wonder if there is something like that in realityView / RealityKit 4. Here is My Code and the screenshots follows

import SwiftUI
import RealityKit

struct ContentView: View {
    
    @State var myHead: Entity = {
        let headAnchor = AnchorEntity(.head)
        headAnchor.position = [-0.02, -0.023, -0.24]
        return headAnchor
    }()

    @State var clicked = false
    
    var body: some View {
        RealityView { content, attachments in
            // create a 3d box
            let mainBox = ModelEntity(mesh: .generateBox(size: [0.1, 0.1, 0.1]))
            mainBox.position = [0, 1.6, -0.3]
            
            content.add(mainBox)
            
            content.add(myHead)
            guard let attachmentEntity = attachments.entity(for: "Dashboard") else {return}
            
            myHead.addChild(attachmentEntity)

        }
        attachments: {
            // SwiftUI Inside Immersivre View
            Attachment(id: "Dashboard") {
                VStack {
                    Spacer()
                        .frame(height: 300)
                    Button(action: {
                        goClicked()
                    }) {
                        Text(clicked ? "⏸️" : "▶️")
                            .frame(maxWidth: 48, maxHeight: 48, alignment: .center)
                            .font(.extraLargeTitle)
                    }
                    .buttonStyle(.plain)
                }
            }
        }
    }
    
    func goClicked() {
        clicked.toggle()
    }
}

Answered by Vision Pro Engineer in 805428022

Hi @ostoura

It is not currently possible to render a reality view attachment such that it always appears in front of other 3D models in the scene. If this is a feature you'd like to see RealityKit support in the future, please file a feedback request at https://feedbackassistant.apple.com and post the FB number here so I can take a look or forward it to the relevant engineers. Thanks!

Additionally, the HIG advises against anchoring UI to a person's head, so you may wish to consider a different approach to designing your dashboard, such as anchoring it in the person's space:

Avoid anchoring content to the wearer’s head. Although you generally want your app to stay within the field of view, anchoring content so that it remains statically in front of someone can make them feel stuck, confined, and uncomfortable, especially if the content obscures a lot of passthrough and decreases the apparent stability of their surroundings. Instead, anchor content in people’s space, giving them the freedom to look around naturally and view different objects in different locations.

That being said, you could replace your dashboard attachment with a model entity and then use the ModelSortGroupComponent to render that entity in front of your other 3D content. I've modified your code to demonstrate how you could go about doing this.

@State var myHead: Entity = {
    let headAnchor = AnchorEntity(.head)
    headAnchor.position = [0, -0.15, -0.4]
    return headAnchor
}()

// Use a model entity to act as a "dashboard" instead of an attachment.
@State var dashboardEntity: ModelEntity = {
    let dashboardEntity = ModelEntity(mesh: .generateSphere(radius: 0.02), materials: [])
    dashboardEntity.generateCollisionShapes(recursive: false)
    dashboardEntity.components.set(InputTargetComponent())
    return dashboardEntity
}()

@State var clicked = false
var clickedMaterial = SimpleMaterial(color: .green, isMetallic: false)
var unclickedMaterial = SimpleMaterial(color: .red, isMetallic: false)

var body: some View {
    RealityView { content in
        // create a 3d box
        let mainBox = ModelEntity(mesh: .generateBox(size: [0.1, 0.1, 0.1]), materials: [SimpleMaterial()])
        mainBox.position = [0, 1.6, -0.3]
        
        content.add(mainBox)
        
        content.add(myHead)
        myHead.addChild(dashboardEntity)

        // Create a model sort group for both entities.
        let group = ModelSortGroup(depthPass: .postPass)
        // Sort the box entity so that it is drawn first.
        let mainBoxSortComponent = ModelSortGroupComponent(group: group, order: 1)
        mainBox.components.set(mainBoxSortComponent)
        // Sort the dashboard entity so that it is drawn second, on top of the box.
        let dashboardSortComponent = ModelSortGroupComponent(group: group, order: 2)
        dashboardEntity.components.set(dashboardSortComponent)
    }
    update: { content in
        // Update the dashboard entity's material when the value of `clicked` changes.
        dashboardEntity.model?.materials = clicked ? [clickedMaterial] : [unclickedMaterial]
    }.gesture(
        TapGesture()
            .targetedToEntity(dashboardEntity)
            .onEnded({ value in
                // Toggle `clicked` when the dashboard entity is tapped.
                clicked.toggle()
            })
        )
}

Here, I've replaced your "Dashboard" attachment with a 3D sphere model entity, but you could use a model that's more suitable for your application, such as a plane with a texture on it or a 3D play/pause button.

The key to this approach is having the dashboard entity and the box entity share a ModelSortGroup with its depthPass set to .postPass, and giving the dashboard entity a higher sorting order than the box entity. This causes the dashboard entity to be drawn last and therefore appear in front of the box, as the documentation describes:

The ModelSortGroup.DepthPass.postPass option tells the renderer to draw entities in reverse order, which gives the effect that the last model it draws appears in front.

Again, refer to the HIG for best practices when designing experiences for visionOS.

Accepted Answer

Hi @ostoura

It is not currently possible to render a reality view attachment such that it always appears in front of other 3D models in the scene. If this is a feature you'd like to see RealityKit support in the future, please file a feedback request at https://feedbackassistant.apple.com and post the FB number here so I can take a look or forward it to the relevant engineers. Thanks!

Additionally, the HIG advises against anchoring UI to a person's head, so you may wish to consider a different approach to designing your dashboard, such as anchoring it in the person's space:

Avoid anchoring content to the wearer’s head. Although you generally want your app to stay within the field of view, anchoring content so that it remains statically in front of someone can make them feel stuck, confined, and uncomfortable, especially if the content obscures a lot of passthrough and decreases the apparent stability of their surroundings. Instead, anchor content in people’s space, giving them the freedom to look around naturally and view different objects in different locations.

That being said, you could replace your dashboard attachment with a model entity and then use the ModelSortGroupComponent to render that entity in front of your other 3D content. I've modified your code to demonstrate how you could go about doing this.

@State var myHead: Entity = {
    let headAnchor = AnchorEntity(.head)
    headAnchor.position = [0, -0.15, -0.4]
    return headAnchor
}()

// Use a model entity to act as a "dashboard" instead of an attachment.
@State var dashboardEntity: ModelEntity = {
    let dashboardEntity = ModelEntity(mesh: .generateSphere(radius: 0.02), materials: [])
    dashboardEntity.generateCollisionShapes(recursive: false)
    dashboardEntity.components.set(InputTargetComponent())
    return dashboardEntity
}()

@State var clicked = false
var clickedMaterial = SimpleMaterial(color: .green, isMetallic: false)
var unclickedMaterial = SimpleMaterial(color: .red, isMetallic: false)

var body: some View {
    RealityView { content in
        // create a 3d box
        let mainBox = ModelEntity(mesh: .generateBox(size: [0.1, 0.1, 0.1]), materials: [SimpleMaterial()])
        mainBox.position = [0, 1.6, -0.3]
        
        content.add(mainBox)
        
        content.add(myHead)
        myHead.addChild(dashboardEntity)

        // Create a model sort group for both entities.
        let group = ModelSortGroup(depthPass: .postPass)
        // Sort the box entity so that it is drawn first.
        let mainBoxSortComponent = ModelSortGroupComponent(group: group, order: 1)
        mainBox.components.set(mainBoxSortComponent)
        // Sort the dashboard entity so that it is drawn second, on top of the box.
        let dashboardSortComponent = ModelSortGroupComponent(group: group, order: 2)
        dashboardEntity.components.set(dashboardSortComponent)
    }
    update: { content in
        // Update the dashboard entity's material when the value of `clicked` changes.
        dashboardEntity.model?.materials = clicked ? [clickedMaterial] : [unclickedMaterial]
    }.gesture(
        TapGesture()
            .targetedToEntity(dashboardEntity)
            .onEnded({ value in
                // Toggle `clicked` when the dashboard entity is tapped.
                clicked.toggle()
            })
        )
}

Here, I've replaced your "Dashboard" attachment with a 3D sphere model entity, but you could use a model that's more suitable for your application, such as a plane with a texture on it or a 3D play/pause button.

The key to this approach is having the dashboard entity and the box entity share a ModelSortGroup with its depthPass set to .postPass, and giving the dashboard entity a higher sorting order than the box entity. This causes the dashboard entity to be drawn last and therefore appear in front of the box, as the documentation describes:

The ModelSortGroup.DepthPass.postPass option tells the renderer to draw entities in reverse order, which gives the effect that the last model it draws appears in front.

Again, refer to the HIG for best practices when designing experiences for visionOS.

Thanks @Vision Pro Engineer for your reply, in fact I was always working with this method to use entity as a dashboard but I never knew there was a sort command like this, thanx again,

Still I would recommend this action be also available for the Attachments because it is more easy to create as a buttons and monitors in games than to build it as 3d entities and also it takes very less memory than an entity dashboard so I would go for a recommend this to be added for the future versions of VisionOS,

About the HIG I would really love to follow and not attach anything to the user head but it would be bit hard for an adventure games which the user need to go from a 3d place to another and imagine if he would like to check his life meter or score or click a button, I think it is not that bad to have a small items on the edge of the user view which is essential for some games like that. but anyway that was great answer and very perfect.

P.S. I Have Added the Feedback as you suggested hoping it would be added to the next VisionOS Version and here is the feedback number: FB15257742

and here is the link to it https://feedbackassistant.apple.com/feedback/15257742

Znear For RealityView to prevent Entity from Hiding an attachment SwiftUI Button in VisionOS
 
 
Q