RealityKit Instanced Rendering on visionOS

Question

VirtDev OP

Created Mar ’24

Replies 2

Boosts 2

Participants 2

Hello,

I've been trying to leverage instanced rendering in RealityKit on visionOS but have not had success.

RealityKit states this is supported:
https://developer.apple.com/documentation/realitykit/validating-usd-files
https://developer.apple.com/videos/play/wwdc2021/10075/?time=1373
https://developer.apple.com/videos/play/wwdc2023/10099/?time=772
RealityKit Trace metrics

Validating instancing is working:
To test I made a base visionOS app with immersive space and the entity replaced with my test usdz file. I've been using the RealityKit Trace profiling template in xcode instruments in the immersive space and volume closed. This gets consistent draw call results.

If I have a single sphere mesh with one material I get one draw call, but the number of draw calls grows linearly with mesh count no matter how my entity is configured.

What I've tried

Create a test scene in blender, export with instancing enabled
Create a test scene in Reality Composer Pro using references
Author usda files by hand based on the OpenUSD spec
Programatically create a MeshResource with Contents at runtime

References
https://openusd.org/release/api/_usd__page__scenegraph_instancing.html
https://developer.apple.com/documentation/realitykit/meshresource
https://developer.apple.com/documentation/realitykit/meshresource/instance

Thank you

Boost

Answer 1

VirtDev OP

Mar ’24

For more context here is a minimal RealityKit view to reproduce the test setup.

    var body: some View {
        RealityView { content in
            let material = SimpleMaterial(color: .systemPink, isMetallic: false)
            let mesh = MeshResource.generateSphere(radius: 1)
            
            // MeshResource.generateSphere creates a model with this name
            let model0 = mesh.contents.instances["MeshModel-0"]
            let modelName = model0!.model
            
            // This implementation tries to create a new MeshResource vs MeshResource.replace()
            var instances: [MeshResource.Instance] = []
            let instanceCount = 20
            for i in 1...instanceCount {
                let translation: SIMD3<Float> = [Float.random(in: 0...1),
                                                 Float.random(in: 0...1),
                                                 Float.random(in: 0...1)]
                // Nothing in the docuementation indicates what the id needs to be.
                let instance = MeshResource.Instance(
                    id: "MeshModel-\(i)",
                    model: modelName,
                    at: Transform(
                        scale: [0.05, 0.05, 0.05],
                        translation: translation).matrix)
                instances.append(instance)
            }
            var contents = MeshResource.Contents()
            contents.models = mesh.contents.models
            contents.instances = MeshInstanceCollection(instances)
            do {
                let finalMesh = try await MeshResource(from: contents)
                var spheres = ModelEntity(mesh: finalMesh, materials: [material])
                content.add(spheres)
            } catch {
                print("failed to create mesh resource \(error.localizedDescription)")
            }
            
        }
    }

Here is a screenshot of the RealityKit trace.

Maybe there is some specific requirements on a usdz file that will trigger RealityKit to used instanced rendering.

0

Answer 2

andershaf OP

Jun ’24

I'm struggling with the same thing, I really need instanced rendering in my app.

0