Metal + UIKit Timing Issues

Question

Created Jan ’22

Replies 5

Boosts 0

Views 2.8k

Participants 4

Hi! I am currently finalizing a new app that uses Metal to render a 3D scene and a UIKit overlay to display controls for interacting with objects in the scene. The render loop is driven via a CADisplayLink with its preferredFramesPerSecond set to 60.

I have recently noticed an issue where the app reports a steady 60 fps frame rate in the Xcode debug navigator, but still felt sluggish on the device. This feeling was only present on devices with ProMotion and often started after interactions with the UIKit overlay. I started investigating by using Metal System Trace and quickly found an explanation for the sluggish feeling: occasionally, the app would switch from its nominal 16ms-16ms-16ms cadence to 12ms-20ms-12ms, thus still averaging 60 fps, but with inconsistent frame times.

Pictures of the timeline can be found here.

I have tried setting the CAMetalLayer's presentsWithTransaction to true, waiting for the command buffer to be scheduled and then presenting the drawable, but, unfortunately, the problem persists.

If anybody can think of a potential reason / solution for this, I would be very thankful.

Boost

Answer 1

Graphics and Games Engineer OP

Apple

Jan ’22

Can you try presenting using -[MTLCommandBuffer presentDrawable:afterMinimumDuration] (or the equivalent MTLDrawable method) with the duration being .016.? This should keep your rendering to 60FPS and in sync with UIKit.

There are some situations where UIKit renders at 120FPS, but this should not be one of them. If you're still seeing issues, please create a Feedback Assistant report (preferably with an example showing what you're seeing) and post it here so we can take a look.

0

Answer 2

vortycon OP

Jan ’22

Thanks for the reply! During the last couple of days, this has turned into quite a mysterious journey. I am now at the point where I don't think it has anything to do with UIKit at all, and might possibly be more widespread.

Here's what I have tried:

presentsWithTransaction = true, maximumDrawableCount = 3, custom Metal view using CADisplayLink set to 60 fps, present(_: ): 12-20-12 cadence occurs sometimes, mostly after interactions with UIKit
presentsWithTransaction = true, maximumDrawableCount = 3, custom Metal view using CADisplayLink set to 60 fps, present(afterMinimumDuration: 0.016): 12-20-12 cadence persists. From my perspective, this is already an indication that something under the hood might be wrong: why is a frame being displayed for 12 ms when the minimum duration is set to 16 ms?

I have then started to take out most of my custom code, starting with replacing my custom Metal view with an MTKView. The problem persisted. I then took out my custom UIKit overlay - still, the problem persisted. I tried switching to double-buffering by setting maximumDrawableCount = 2 - now, the 12-20-12 cadence seems to occur all the time, not just after interactions with UIKit.

Today, I wanted to try a different approach and started from a blank Xcode project. I added an MTKView, set preferredFramesPerSecond = 60 and: the 12-20-12 cadence showed up again(!).

I have uploaded the project here.

I have also noticed that when I set the minimum duration to 0.016 when double-buffering, the frames are being displayed for much longer (around 250 ms). The two problems might be unrelated, but this behavior can also be found in the sample project and I wanted to at least mention it here.

FB9860634

0

Answer 3

vortycon OP

Nov ’22

I wanted to follow up with a quick update, since quite some time has passed since I first made this post.

Sadly, the problem still exists on all of my test iPads with ProMotion, updated to iPadOS 15.7, in my eyes significantly downgrading the user experience for apps targeting 60 fps. The 12-20-12 cadence can be observed via Metal System Trace with my minimal sample project uploaded here. The problem disappears as soon as the display refresh rate is limited "externally" by Low Power Mode, making the app run noticeably smoother. Interestingly, I have yet to see the issue on iPhones with ProMotion.

I can see that there are probably more urgent problems to solve, but would be really thankful if this issue could be looked at again.

0

Answer 4

alkjslkjslkjds OP

Aug ’23

This is still happening on MacBook Pro M2 Max with ProMotion enabled (but not when refresh rate is fixed to 60 FPS). There is a bug in ProMotion somewhere (when it's in full screen mode only). Setting afterMinimumDuration doesn't help.

0

Answer 5

alkjslkjslkjds OP

Aug ’23

My answer doesn't include code, but I believe it will be helpful as I spent 3 days before I figured it out.

You need to use a semaphore to control access to the "in-flight" frames, and the maximum of the "in-flight" frames should be equal to 3 (because MTKView uses triple buffering).

To find out how to do this correctly is easy. Start a new "Game" project in xcode, and select the framework "Metal" (not "SceneKit" or "SpriteKit"), and you can skip tests generation. Then you can find the relevant code in the "Renderer.swift" file (or whatever language you selected).

I commented out all the "cube rotation" rendering in the default project, and just left the drawable presentation. The glitch disappeared at 60 FPS when the window is maximised (or not). Additionally I tried rendering the full video sample with graphics at 120 FPS (by setting preferredFrameRate on the view), and it works marvellously. You can enable metal information display in the options to see that there is no fluctuation if frame rate. Well, there is, but not glitchy anymore.

I'm using MacBook Pro M2.

0