How to programmatically update Model Position Offset of GeometryModifier?

Question

ckse93 OP

Created 4d

Replies 1

Boosts 0

Participants 2

is it possible to dynamically update ModelPositionOffset of GeometryModifier with a depth map image?

in my code I set up the parameter for "DepthMapTexture" universal input node Screenshot 2025-03-09 at 12.00.25 AM.png

and tried setting the depth map for depthTextureResource. I have 2 DrawableQueues. One for setting InputTexture, and one for setting DepthMapTexture. This only shows the part that concerns setting DepthMapTexture Screenshot 2025-03-09 at 12.02.35 AM.png

this is where I define the plane entity. Screenshot 2025-03-09 at 12.06.03 AM.png

and this is the shader graph Screenshot 2025-03-08 at 11.58.49 PM.png

what I noticed with GeometryModifier is that, the depthMap image has to be same as input image's dimensions. and when I applied this material to usdz file, with pre-assigned image and depth map from RCP, and loaded that Entity from code, depth map was applied correctly.

what I am unsure is that if it is impossible to define a model entity from code, apply ShaderGraphMaterial from RCP, and dynamically update the image used in GeometryModifier. Maybe I'm missing something when defining Entity, something that allows geometric modifications?

Boost

Answer 1

Vision Pro Engineer OP

Apple

1d

Hi @ckse93

It should absolutely be possible to dynamically update the ModelPositionOffset of GeometryModifier node with a depth map image!

I believe the issue you're running into is due to a lack of vertices in the plane mesh generatePlane(width:depth:cornerRadius:) creates (you can visualize this by setting the triangleFillMode of the material applied to your plane mesh to lines, which will render the wireframe of your mesh).

For context, the GeometryModifier node operates on each individual vertex in a mesh. This means that you ideally want your mesh to have the same dimensions as your depth map image, so that each pixel in the depth map corresponds to a vertex in the mesh. For example, if your depth map image is 512x256 pixels, you'll want a plane mesh that's 512x256 vertices.

To dynamically create a mesh with the same dimensions as your depth map, take a look at the Creating a plane with low-level mesh article. It demonstrates how to dynamically create a plane mesh of any size/dimensions with the LowLevelMesh API. In fact, that article is related to a broader sample project, Generating interactive geometry with RealityKit, which demonstrates how to create a dynamic depth and normal map with LowLevelTexture and may be relevant to the kind of effect you're trying to create here.

If you'd like to see generatePlane(width:depth:cornerRadius:) provide an option to generate a plane with specified dimensions, please file a enhancement request with Feedback Assistant.

Let me know how this all goes!

0