Display/constraint update cycle crashes in macOS 11/12

I recently migrated to macOS Monterey to build my macOS app. After building the app in 12.3, the users have started to experience mysterious crashes related to view layout.

The first one was somehow related to NSScrollView endlessly resizing itself between two width values, and I figured it could have something to do with scroll bar sizing, but found no apparent fix.

Crashing on exception: The window has been marked as needing another Display Window pass, but it has already had more Display Window passes than there are views in the window.

I circumvented the issue with some duct tape, but ended up with another layout exception on other users' systems:

Crashing on exception: The window has been marked as needing another Update Constraints in Window pass, but it has already had more Update Constraints in Window passes than there are views in the window.

It's very hard to debug this, because the bug seems to be very system-specific.

However, by intense googling, I've seen that Safari and Xcode have also suffered from the exact same crash, but I've found no information on how they might have been fixed.

I'm starting to suspect that there is something principal that has been changed within view layout routines macOS 11/12 that I'm completely missing. Does anyone have any experience with such issues, and how to start digging deeper?

Similar issue is discussed here: https://developer.apple.com/forums/thread/114579 It hints to something about changes in layer backing, but no further information is available.

It appears that this is related to scaled resolution. On some systems, default scaling causes the issue, while on others you can set any scaling and never experience a crash.

A trick which fixes this crash on all systems is to subclass NSClipView, override its layout method and never call super methods. Just don't layout the NSClipView. I'm not sure if this will cause further damage along the way, but works as a quick and dirty fix to crashes. Mileage may vary.

To anyone wondering about this:

Something in view layout pipeline was changed in Big Sur or Monterey so that setting the frame of document view will cause NSClipView to invalidate its layout. I had overridden setFrame in my NSTextView, and as the frame was set, NSClipView would cause an infinite loop by calling setFrame again.

Display/constraint update cycle crashes in macOS 11/12
 
 
Q