Here is a sample asset demoing the problem:
https://cloudzeta.com/zeta/public-demo/session/lx9tkmenrcj4o5ad/quicklook.usdz
Assets like this used to work well in visionOS 1, but now it's missing textures for some reason:
https://www.youtube.com/watch?v=2TOMnkGvi8I
Our engineering teams need to investigate this issue, as resolution may involve changes to Apple's software. I'd greatly appreciate it if you could open a bug report, include the USDZ and video, and please post the FB number here once you do.
Bug Reporting: How and Why? has tips on creating your bug report.