RealityKit Systems Not Ticking Every Frame on visionOS

Please see also the video demo of the problem I'm encountering: https://youtu.be/V0ZkF-tVgKE

I've noticed that the custom Systems I've been creating for my RealityKit/visionOS app do not get updated every frame as the documentation (and common sense) would suggest. Instead, they appear to tick for a time after each UI interaction and then "stall". The systems will be ticked again after some interaction with the UI or sometimes with a large enough movement of the user. My understanding was that these Systems should not be tied to UI by default so I'm a bit lost as to why this is happening.

I've reproduced this by starting from a template project and adding a very simple couple of systems.

Here is the main System, which simply rotates the pair of spheres:

import RealityKit
import RealityKitContent
import SwiftUI

public struct RotationSystem: System {

    static let query = EntityQuery(where: .has(RealityKitContent.WobblyThingComponent.self))

    public init(scene: RealityKit.Scene) {
    }

    public func update(context: SceneUpdateContext) {
        
        print("system update, deltaTime: \(context.deltaTime)")
        
        let entities = context.scene.performQuery(Self.query).map({ $0 })

        for entity in entities {
            
            let newRotation = simd_quatf(angle: Float(context.deltaTime * 0.5), axis: [0, 1, 0]) * entity.transform.rotation
                    
            entity.transform.rotation = newRotation
        }
    }
}

The component (WobblyThingComponent) is attached to a parent of the two spheres in Reality Composer Pro, and both system and component are registered on app start in the usual way.

This system runs smoothly in the simulator, but not in the preview in XCode and not on the Vision Pro itself, which is kinda the whole point.

Here is a video of the actual behaviour on the Vision Pro: https://youtu.be/V0ZkF-tVgKE

The log during this test confirms that the system is not being ticked often. You can see the very large deltaTime values, representing those long stalled moments:

system update, deltaTime: 0.2055550068616867
system update, deltaTime: 0.4999987483024597

I have not seen this problem when running the Diaroma sample project, yet when comparing side-by-side with my test projects I cannot for the life of me identify a difference which could account for this.

If anyone could tell me where I'm going wrong it would be greatly appreciated as I've been banging my head against this one for days.

Xcode: Version 15.3 (15E204a) visionOS: 1.1 and 1.1.1

Accepted Reply

Instead of context.scene.performQuery did you try context.entities(matching: query, updatingSystemWhen: .rendering)?

  • @arthurfromberlin I tried that just now and IT WORKS! 🥳

    But... can you explain in more depth WHY that works? I'm familiar with scheduling of systems from Bevy, so I understand that systems might be ticked at different times, but the query happens INSIDE the system, after my console log no less. I really don't understand why that would dictate when the system's update function is called.

    Thank you so much for the tip!

  • Glad it worked! 😊 They just added this method and system update condition stuff on visionOS. On iOS afaik systems always seem to be synced with the render loop by default. My assumption would be that since visionOS has high and variable refresh rates and there is certain logic that does not necessarily need to run on every frame, they added SystemUpdateCondition to make the API more flexible in the future: https://developer.apple.com/documentation/realitykit/systemupdatecondition

  • Interesting! I'm totally new to RealityKit (long time Unity dev) and was wondering if it was something like this. But it seems the docs insist "RealityKit calls every system’s update(context:) method every frame". Probably the docs need updating with visionOS, eh?

Replies

Instead of context.scene.performQuery did you try context.entities(matching: query, updatingSystemWhen: .rendering)?

  • @arthurfromberlin I tried that just now and IT WORKS! 🥳

    But... can you explain in more depth WHY that works? I'm familiar with scheduling of systems from Bevy, so I understand that systems might be ticked at different times, but the query happens INSIDE the system, after my console log no less. I really don't understand why that would dictate when the system's update function is called.

    Thank you so much for the tip!

  • Glad it worked! 😊 They just added this method and system update condition stuff on visionOS. On iOS afaik systems always seem to be synced with the render loop by default. My assumption would be that since visionOS has high and variable refresh rates and there is certain logic that does not necessarily need to run on every frame, they added SystemUpdateCondition to make the API more flexible in the future: https://developer.apple.com/documentation/realitykit/systemupdatecondition

  • Interesting! I'm totally new to RealityKit (long time Unity dev) and was wondering if it was something like this. But it seems the docs insist "RealityKit calls every system’s update(context:) method every frame". Probably the docs need updating with visionOS, eh?