USDZ with Blend Shapes Workflow Recommendations

Hi @arthurfromberlin !

Yes indeed RealityKit does support blend shapes! Unfortunately we don't have official guidance for workflows in third party DCCs like Blender or Cinema4D (btw, the link you shared has 404'd for me). I am not an expert on these software packages by any measure, but my educated guess is that each contains similar functionality for exporting blend shape content to USD format, and I absolutely recommend reaching out to other communities for further guidance on DCC-specific workflows.

I'll speak a bit on how I've made assets in Blender 4.2, in case you or anyone reading is using that. The terminology Blender 4.2 uses for blend shapes is "Shape Keys." When you export a USD asset from Blender 4.2, make sure you are including Shape Keys in your export under the "Rigging" section. Additionally, disable "Convert World Material" under the "Materials" section.

Once you have your USD with blend shapes, you can load it in as a ModelEntity that will have a BlendShapeWeightsComponent. This will allow you to control your entity's blend shapes. In the following example, I created an asset with a single weightSet and two blend shapes, and set the first blend shape to have complete influence over the original shape:

if let cube = try? await ModelEntity.init(named: "blend_shape_demo", in: Bundle.main) {
    var blendShape = cube.components[BlendShapeWeightsComponent.self]!
    
    // weightSet: Assets can have multiple weight sets for different parts of their
    //   model, such as the facial features of a character with many facial
    //   expressions. In this example I am only using the
    //   first weightSet, which contains two blend shapes.
    
    // weights: Each weightSet has a weight value for each blend shape in the set.
    //   Weight values are a numerical representation of the impact a blend shape has
    //   on the original shape. In this example, the first blend shape will have
    //   complete influence over the original shape.
    
    blendShape.weightSet[0].weights = [1.0, 0.0]
    
    // don't forget to set the component and add the entity to the scene!
    cube.components.set(blendShape)
    content.add(cube)
}

Let me know if you have any questions! Be sure to check out Compose interactive 3D content in Reality Composer Pro for more info on blend shapes, as well as the attached sample project containing a working demo of blend shapes in RealityKit.

@Vision Pro Engineer

I tried to load a usdz created using blender in xCode. All blend shapes are detected but not all can be updated. When I checked the values of blend shape weight set, some of the weight names are blank, and these blank weight names are the blend shapes which do not reflect the weight update.

Example: I have 7 shape keys (blend shapes equivalent in blender) in my exported usdz model in blender In xCode, the following can be traced from the loaded model:

_weightNames =

[0] = (String) "\u{2}"

[1] = (String) ""

[2] = (String) ""

[3] = (String) ""

[4] = (String) "blendshape1"

[5] = (String) "blendshape2"

[6] = (String) "blendshape3"

items [0]-[3] has incorrect/blank names

It seems that weight names are arranged alphabetically, but only the last few ones can be detected. Are there any needed specification with usdz model? Note: When the usdz I loaded is exported to SceneKit using xCode, all blend shapes can be updated.

Thank you!