Yep - same problem - no solution. Currently in the too difficult box. The big elephant in the room is that there are two systems and you have to fight one too get the features from the other - world save/load for example. I keep wondering whether to drop RealityKit as it still hasn't caught up with ARKit in a number of vital areas. The other big issue is that there is no Model I/O to RealityKit mechanism. I had to write my own in the end and the documentation for custom meshes etc for RealityKit is genuinely non-existent!
Where am I going with this? Oh Yes.
I really hope that RealityKit 3 deals with a lot of this - when you try to do real work with it there are some howling gaps.