I have a model that uses a video material as the surface shader and I need to also use a geometry modifier on the material.
This seemed like it would be promising (adapted from https://developer.apple.com/wwdc21/10075 ~5m 50s).
// Did the setup for the video and AVPlayer eventually leading me to
let videoMaterial = VideoMaterial(avPlayer: avPlayer)
// Assign the material to the entity
entity.model!.materials = [videoMaterial]
// The part shown in WWDC: Set up the library and geometry modifier before, so now try to map the new custom material to the video material
entity.model!.materials = entity.model!.materials.map { baseMaterial in
try! CustomMaterial(from: baseMaterial, geometryModifier: geometryModifier)
}
But, I get the following error
Thread 1: Fatal error: 'try!' expression unexpectedly raised an error: RealityFoundation.CustomMaterialError.defaultSurfaceShaderForMaterialNotFound
How can I apply a geometry modifier to a VideoMaterial? Or, if I can't do that, is there an easy way to route the AVPlayer video data into the baseColor of CustomMaterial?
Hi, I looked at the RealityKit source code, and we only support geometry modifiers on PhysicallyBasedMaterial, UnlitMaterial, and SimpleMaterial. So you will have to route the AVPlayer video data to a CustomMaterial texture. You might be able to accomplish this using the DrawableQueue API. However, I don't know how you can actually get the AVPlayer data into the DrawableQueue, but I'm sure it's possible.
Please file a feature request for this on Feedback Assistant and post the feedback number here, I will send the request to the RealityKit graphics team. If you could provide some more info about your use case in the feature request that would be great.
