RealityKit add gestures and animations to Entity/ModelEntity

I've been looking at a similar problem over the last day or so. I had a look in SwiftStrikeTabletop and the in there they cast an Entity from a Reality Composer scene to Entity & HasCollision. Here's a sample I wrote :

// Setup AR Session
let arConfiguration = ARWorldTrackingConfiguration()
arConfiguration.planeDetection = .horizontal
arView.session.run(arConfiguration)
// Load the "Box" scene from the "Experience" Reality File
let boxScene = try! Experience.loadBox()
boxScene.generateCollisionShapes(recursive: true)
let box = boxScene.greybox as? Entity & HasCollision
arView.installGestures(for: box!)

This seems to work (although the Rotation gesture doesn't currently appear to respond).

+1

Hi hulbrich,

did you find a solution for this? davidjohnston´s solution (simply casting to Entity & HasCollsion) doesn´t work for me. I have the same problem, even though I am loading the models from the filesystem after downloading from a server:

• Entity.loadAsync() works for the embedded animation in a USDZ.

• ModelEntity.loadModelAsync() works for adding gestures and anything else I am doing with the objects in my scene. Animations are lost this way since apparently living under Entity not ModelEntity.
Any input is appreciated.

Thanks

Timo

Hello all,

Here is a detailed snippet that will get you the desired behavior. It will repeat all animations, while still allowing you to use the standard gestures to manipulate the model.

loadRequest = Entity.loadAsync(contentsOf: url).sink(receiveCompletion: { status in
            print(status)
        }) { entity in
           
            // Scaling entity to a reasonable size
            entity.setScale(SIMD3(repeating: 0.01), relativeTo: nil)
           
            // Creating parent ModelEntity
            let parentEntity = ModelEntity()
            parentEntity.addChild(entity)
           
            // Anchoring the entity and adding it to the scene
            let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: .zero))
            anchor.addChild(parentEntity)
            self.arView.scene.addAnchor(anchor)
           
            // Playing availableAnimations on repeat
            entity.availableAnimations.forEach { entity.playAnimation($0.repeat()) }
           
            // Add a collision component to the parentEntity with a rough shape and appropriate offset for the model that it contains
            let entityBounds = entity.visualBounds(relativeTo: parentEntity)
            parentEntity.collision = CollisionComponent(shapes: [ShapeResource.generateBox(size: entityBounds.extents).offsetBy(translation: entityBounds.center)])
                       
            // installing gestures for the parentEntity
            self.arView.installGestures(for: parentEntity)
           
        }
       
        // Visualize the collision components
        arView.debugOptions = .showPhysics

The key here is to nest the loaded entity which has the animations inside of a ModelEntity, and to then give that ModelEntity an appropriate CollisionComponent based on the loaded model's bounds.

Unfortunately this doesn´t work for me 😟

By setting arView.debugOptions = .showPhysics, the gestures (scale, rotate, translate) work only for the collision shape itself (parentEntity) and not for 'entity'!

Thanks,

As a child of parentEntity, entity should appear to be scaled, rotated, and translated when parentEntity is scaled, rotated, and translated.

I'm getting the same results as Blueplanet. The parentEntity correctly responds to the gestures, but the child entities (in my case) just stay stuck to the same spot. Any chance this is a bug?

I was thinking maybe there is a delegate I need to watch and then individually update all the child entity positions, but seems like too much work for something simple.

Thanks! It works, even the rotation too. Though it's needed to

set Phisics for the object in Reality Composer.

Thanks gchiste, your solution works!

Thank you gchiste. Your solution worked for me

Doesn't work for visionOS where there is no ARView :-/