Redraw MTKView when its size changes

I am new to Metal and must therefore be missing something obvious here. I am using an MTKView to present some content. It all works very well, except that I get some jerky motion when the view is resized. The problem is that I don't want the view to simply scale its content when it is resized, but instead I would like to redraw the content entirely myself. I have set paused and enableSetNeedsDisplay to YES, so the view only gets redrawn when needed (the content is not animated, but it changes in response to user action and I need to take advantage of the GPU to make this responsive), but my delegate's drawInMTKView: method only gets called very occasionally when the window (and therefore the MTKView) is resized.


In between these calls, it seems that the MTKView simply rescales its content, which then yields jerks whenever the window content gets refreshed. The mtkView: drawableSizeWillChange: method seems to be called more often during the resize, so I tried directly calling the view's draw method from there. This works a bit better, but still leads to jerks, only more often and less pronounced. It also somehow doesn't feel like the right thing to do anyway...

Replies

Try setting MTKView.autoResizeDrawable to NO (this is YES by default). The drawable will not resize so rendering will get blurrier as the window gets bigger (since there will be less pixels in the drawable than pixels in the window). When the resize finishes completely, you can set the drawableSize to something that matches the windowsize a little better.

Thanks a lot, but this is precisely what I am trying to avoid. I do not want the content to be simply rescaled when the size changes, but instead I would like to have the opportunity to rerender the content myself as soon as the size changes.

I have the same problem and it drives me crazy. Apple suggests combining CAMetalLAyer.presentsWithTransaction = YES, and and using the [MTLCommandBuffer waitUntilScheduled] method, along with [Drawable present]:


https://developer.apple.com/documentation/quartzcore/cametallayer/1478157-presentswithtransaction?language=objc


I also do not want my content to be stretched during live resize, I would like my UI to lay out its contents properly. But the flickering just gives you a headache. The only workaround that remotely helps, is to make the metal layer bigger than you'll ever need it, and don't change its size, but even this produces has some flickering. And also with this approach you can see your content lagging the window edges as frames are rendered for a window size that was correct 30ms ago.


Can somebody at Apple please advise?

Hi jzrake,


What isn't working for you either with Dan's suggestion or the alternative suggestion that you mentioned?

Hi 4k4,


Thanks for your reply. I do adopt Dan's suggestion, to manage the drawable size manually. The options are either:


1. Fix the drawable size during live resize, and change it after resizing is finished. The content stretches and there is no flickering. However the contents are then scaling rather than being laid out properly. It's maybe OK for a game, but not for a UI program.


2. Fix the drawable size *always*, to be larger than you expect the window will ever get. There is reduced flickering but the rendering lags the window edges.


Neither option looks professional, and the flickering is definitely not acceptable. I believe the solution must involve ensuring the rendering is sync'ed to the resize, such that the window cannot resize again until the previous frame is finished rendering. However, I see bad flickering even when I force the drawing to block in this way.


To be clear, the HelloTriangle program provided by Apple *does* exhibit this problem, you just need to move the triangle (see edit below) to one of the window edges in order to see it happening (it's not as evident when it's in rendered the middle). If somebody at Apple could update the HelloTriangle program with a solution, many people could use that as a guide.


Thanks!



EDIT:


In the AAPLShaders.metal source file from HelloTriangle.xcodeproj, make the following change on line 57:


// out.clipSpacePosition.xy = pixelSpacePosition / (viewportSize / 2.0);

out.clipSpacePosition.xy = -1 + pixelSpacePosition / (viewportSize / 2.0);


https://developer.apple.com/documentation/metal/hello_triangle?language=objc

Thank you for the clarifications. We're looking into this.

Really glad to hear it's being looked into. I'm happy to send my version of HelloTriangle that highlights the problem if you're having any trouble reproducing it. I should also mention that this problem seems apparent both with and without use of the MTLView class. In production I don't use it, I manage the rendering cycle and Metal layer(s) manually. That extra degree of control allows me to initiate rendering cycles precisely in response to resize events, and then to block the message loop until frames are finished. However, as I mentioned before that procedure does not eliminate the flickering.


Thanks again!

Is Apple still looking into this issue? Note that I updated my previous post with more info on reproducing the problem.

Yes, still looking into it and there are active conversations focused on finding a solution.

Could you give me a clue how Apple is trying to fix the problem so that I might experiment on my own?

Would you guys have any news on this? I'm running into the exact same trouble with an image app using Metal in a CAMetalLayer: whenever I resize the Metal layer, I get alternating stretches and Metal redraws, and that's really ugly 😟

I figured out how to make a Metal layer that resizes with the window smoothly and without glitches. It involves using a `CAMetalLayer`, `presentsWithTransaction` and a few other things. I wrote a blog post about it where I link to some working sample code http://thume.ca/2019/06/19/glitchless-metal-window-resizing/

I'm working on an app that displays a MTKView and a NSTextView in a split screen. When the TextView is empty live resize works just fine, rescaling the MTKView real time. But when the TextView contains a lot of text such that the scroll bar is active the MTKView doesn't receive redraw commands during live resize. The image just streches during the resize and doesn't display correctly until the resize is over. So in my case I would say the problem is with the NSTextView or the NSScrollView which is doing something which blocks the refresh of the MTKView.