How to use NSScrollView with a very large MTKView document view?

We don't have any GPU families that support textures greater than 16384 x 16384 in size, so I recommend trying your second option. You can put the MTKView inside of another NSView (see the following structure) and then synchronize its frame.

Code Block 
NSScrollView
	NSClipView
		NSView
			MTKView

I recommend reading https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/NSScrollViewGuide/Articles/SynchroScroll.html. It's related to synchronizing two scroll views, but you should be able to adapt it and synchronize the location of the MTKView within the parent NSView.

Sorry! Please try this link instead:
https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/NSScrollViewGuide/Articles/SynchroScroll.html

That is a great suggestion- it didn't occur to me to place a subview into the document view. I've worked with the scroll sync code before which is pretty straightforward. I have a feeling that a lot of NSRect-math will be involved, but I think the result will be pretty performant. Thanks!

Hi Demitri,

I was in the same position: I shipped an app last year that embedded a MTKView in an NSScrollView. To work around the texture size limit, I set the maximum zoom on the scroll view to never exceed the metal texture limits.

This year, I implemented what you are now trying to implement: I have an NSScrollView that's synced to a metal view. The scroll view's content is set to the image size and handles all the panning, zooming, etc. The metal view is never resized; it sits below the scroll view and is updated when the scroll view changes. This was a bit of work to implement, but the results are very good.

I took the time to create an intermediary: I first created a Metal-backed view (an NSView backed by a CAMetalLayer) that can display a arbitrary CIImage. I then added scale and offset settings to this view, which perform a CIImage transform and CIImage crop. This essentially creates a region of interest for the CIImage, and just that region is rendered to a Metal texture.

Once this intermediary control was created, it was simple to sync it with the NSScrollView.

Happy to go into more detail, if required.

Hi tinrocket -

I was just about to post an update on my progress! I would greatly appreciate more detail. This is where I have got to:

As suggested above, I created a subclass of NSView to be the scroll view's document view and placed a MTKView as its subview (let's call it the image view). To keep the texture to within the memory limits, the image view is set to be the smaller of the image*scale size or the visibleRect size within the clip view. The document view is always resized to image*scale to make the scroll view work.

To keep the image view in place I placed two constraints at the top and leading of the document view; these are adjusted as the sizes change. (Basically it floats around the empty document view.) I calculate what part from my original image is visible and draw that. I can now magnify indefinitely. This works, but it's not great:

• I get a lot of "tearing" when the magnification is greater than the visible rect size and I scroll around, i.e. the newly exposed areas are black and then get filled in.

• I am not seeing the benefits of responsive scrolling / overdraw.

• I feel like I'm effectively reimplementing NSClipView.

I reengineered it again to remove the document view and make my MTKView the document view. This time, I made its size image*scale and hoped that the dirty rect passed into drawRect: would effectively do the same as above, but this time with the benefits of responsive scrolling (which I specifically opted into), but I was always being asked to redraw the full view (so back to memory problems).

I would love to learn more about how you accomplished this. The last part of my render pipeline is displaying a CIImage, so it should match yours. An Apple engineer in a WWDC20 lab suggested looking at MTLViewport but I couldn't figure out how this would work with a CIRenderDestination.

Thanks for any help!

Hi Demitri,

I did another deep dive into the problem and can share a resuable control and test harness:

https://github.com/Tinrocket-LLC/TRMetalImageView

It's fast, because it only draws the visible region of the scroll view's .documentView, and handles all the edge cases I've encountered in deploying this control, in my commercial apps, over the past two years.

Hope this helps!

John