How to draw a mesh generated by SceneReconstructionProvider in VisionOS

Question

Created Aug ’23

Replies 3

Boosts 2

Participants 3

I want to draw a mesh generated by SceneReconstructionProvider with a material set, but how should I draw the ShapeResource? I want to give a material to the mesh of the recognized wall or object to direct it.

print("Part of the scene has been updated: ", update.anchor)
            task(priority: .low) {
                let shape = try await ShapeResource.generateStaticMesh(from: update.anchor)
                let entity = Entity()
                // What do I need to add?
                entity.components[CollisionComponent.self] = .init(shapes: [shape])
                entity.components[PhysicsBodyComponent.self] = .init(
                    massProperties: .default, .
                    material: nil, ,
                    mode: .static)
            }
```

Boost

Answer 1

adsfadsfsdf OP

Feb ’24

Did you ever figure this out?

0

Answer 2

pj4533 OP

Feb ’24

Curious about this also.

From the SceneReconstructionExample there is this code:

let entity = ModelEntity()
entity.transform = Transform(matrix: meshAnchor.originFromAnchorTransform)
entity.collision = CollisionComponent(shapes: [shape], isStatic: true)

I think you need to add this line:

entity.model = ModelComponent(mesh: <#T##MeshResource#>, materials: <#T##[Material]#>)

Just not sure how to make a MeshResource out of the MeshAnchor (or the ShapeResource that generateStaticMesh returns).

0

Answer 3

pj4533 OP

Feb ’24

Went deep in google and turned up a project from a few months ago that shows converting the MeshAnchor.Geometry to a ModelEntity! 🎉

Here is the link: https://github.com/XRealityZone/what-vision-os-can-do/blob/main/WhatVisionOSCanDo/ShowCase/WorldScening/WorldSceningTrackingModel.swift#L70

And here is the relevant code:

@MainActor fileprivate func generateModelEntity(geometry: MeshAnchor.Geometry) async throws -> ModelEntity {
        // generate mesh
        var desc = MeshDescriptor()
        let posValues = geometry.vertices.asSIMD3(ofType: Float.self)
        desc.positions = .init(posValues)
        let normalValues = geometry.normals.asSIMD3(ofType: Float.self)
        desc.normals = .init(normalValues)
        do {
            desc.primitives = .polygons(
                // 应该都是三角形，所以这里直接写 3
                (0..<geometry.faces.count).map { _ in UInt8(3) },
                (0..<geometry.faces.count * 3).map {
                    geometry.faces.buffer.contents()
                        .advanced(by: $0 * geometry.faces.bytesPerIndex)
                        .assumingMemoryBound(to: UInt32.self).pointee
                }
            )
        }
        let meshResource = try await MeshResource.generate(from: [desc])
        let material = SimpleMaterial(color: .red, isMetallic: false)
        let modelEntity = ModelEntity(mesh: meshResource, materials: [material])
        return modelEntity
    }

3