Non-convex collision?

@JayDev85 Are you able to access the mesh vertices, indices, etc? Worst case you could create convex meshes from the concave mesh yourself.

Are you able to access the mesh vertices, indices, etc with the API? Worst case you could create convex meshes from the concave mesh yourself.

so...

If you have a ModelEntity, you can get the .model which contains a mesh (https://developer.apple.com/documentation/realitykit/modelcomponent/mesh)

The mesh contains this contents struct (https://developer.apple.com/documentation/realitykit/meshresource/contents-swift.struct) , and eventually all the way down to the MeshResource.Part parts (https://developer.apple.com/documentation/realitykit/meshresource/part). Then you start to have index / triangle data.

By this point I feel pretty in-over-my-head....

What has worked is simply going into my scene in RealityComposer and creating a bunch of cubes with collision / physics, and placing them around the model / setting a transparent material. If you're willing to spend 3 hours lining your scene with cubes manually, this does work for simple concave-ish geometries.

There has to be a better solution...

The solution, I think, is to decompose the concave mesh into convex meshes. Then, if this is a static mesh, then you're in luck because optimal performance doesn't matter much if you just want a result in a reasonable amount of time. Resave it as a collection of meshes for reloading in the future. If it's a dynamic mesh, you're kind of stuck doing this at runtime. I think this is a very normal thing to do. Concave collision detection is more expensive than convex.

Note: I think it would be useful to include some built-in algorithm for handling this both in Metal Performance and RealityKit APIs. (Maybe file a feature request?)

In general, decomposing a concave object into a set of convex objects is difficult. I.e. it's NP-Hard. There is lots of literature about this, including some approximation algorithms. Search the web.

But I note that the original question referred to "terrain". Is this concave object terrain? That is an interesting special case which may be easier to solve.

Note that I know next-to-nothing about ARKit / RealityKit, but I do know a fair bit about geometry algorithms in general.

The standard way to use a concave collision shape in RealityKit would be to use ShapeResource.generateStaticMesh. New in 2024, it's easier than ever to use generateStaticMesh by converting a MeshResource to a ShapeResource. Keep in mind that this is a computationally intensive operation, and it has some limitations. In particular, an Entity can not move in response to physics forces (e.g. gravity) when it contains a generateStaticMesh-based shape.

Another approach is to use convex proxies (via ShapeResource.generateConvex, generateBox, and so on). As helpfully mentioned by @endecotp, the decomposition of a concave 3D shape into convex collision proxies is a nontrivial and computationally intensive problem. Ideally, you would perform an approximate convex decomposition ahead of time to save power and improve performance. There are open source solutions which can help you with this such as v-HACD.

Finally, if there are specific features or APIs you would like to see in RealityKit in the future, it would be helpful to us if you provided details in Feedback Assistant.