Creating and applying a shader to change an entity's rendering in RealityKit

Question

peggers123 OP

Created 14h

Replies 1

Boosts 0

Participants 2

Hello,

I am looking to create a shader to update an entity's rendering. As a basic example say I want to recolour an entity, but leave its original textures showing through:

Screenshot 2024-10-17 at 15.02.39.png

I understand with VisionOS I need to use Reality Composer Pro to create the shader, but I'm lost as how to reference the original colour that I'm trying to update in the node graph. All my attempts appear to completely override the textures in the entity (and its sub-entities) that I want to impact. Also the tutorials / examples I've looked at appear to create materials, not add an effect on top of existing materials.

Any hints or pointers?

Assuming this is possible, I've been trying to load the material in code, and apply to an entity. But do I need to do this to all child entities, or just the topmost?

do {
    let entity = MyAssets.createModelEntity(.plane) // Loads from bundle and performs config
    let material = try await ShaderGraphMaterial(named: "/Root/TestMaterial", from: "Test", in: realityKitContentBundle)
    entity.applyToChildren {
        $0.components[ModelComponent.self]?.materials = [material] 
    }
    root.addChild(entity)
} catch {
    fatalError(error.localizedDescription)
}

Boost

Answer 1

Vision Pro Engineer OP

Apple

10h

Hi @peggers123

You can recolor an entity without removing its textures by editing the custom Shader Graph material. The key is to multiply the texture color in the shader by your new color. Let me walk you through how you can do this with the biplane model, like you show in your post.

In Reality Composer Pro:

Add the "Toy Biplane" model to your scene.
Open the biplane's Shader Graph material. I believe it's named "usdpreviewsurface3sg".
Locate the "file5_mtlx" Image node and make note of how its output is connected to the Diffuse Color input of the Preview Surface node. You want to intercept this output and multiply it by your new color before it connects to the Diffuse Color input.
So, add a Multiply node.
Connect the Out parameter of the "file5_mtlx" Image node to the top input of the Multiply node, and connect the output of the Multiply node to the Diffuse Color input of the Preview Surface node.
Add a new Uniform Input to the material of type Color3 (Float) and connect it to the bottom of the Multiply node. Name this input "Color".
Change the value of the Color input to recolor the model.

You can also update the color of the model programmatically In Xcode by obtaining a reference to the ShaderGraphMaterial and setting its Color parameter:

RealityView { content in
    // Add the initial RealityKit content
    if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
        
        // Get the biplane entity and its Shader Graph material.
        if var biplaneEntity = immersiveContentEntity.findEntity(named: "toy_biplane_base_realistic_lod0"),
           var biplaneMaterial = biplaneEntity.components[ModelComponent.self]?.materials[0] as? ShaderGraphMaterial {
            // Set the color parameter.
            try? biplaneMaterial.setParameter(name: "Color", value: .color(.red))
            // Re-apply the material.
            biplaneEntity.components[ModelComponent.self]?.materials[0] = biplaneMaterial
        }
        
        content.add(immersiveContentEntity)
    }
}

This code assumes you added the biplane model to the default "Immersive" scene that comes with your project, but you can adapt it to load a different model in a different scene if needed.

0