`MTKView` on visionOS

Not an engineer, but my guess is that using Metal at the moment is really different to set-up and doesn’t involve using a straight UI/NSView-style application. Rather, you call into compositorservices as in this talk: https://developer.apple.com/videos/play/wwdc2023/10089/

My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds.

In the view controller's view

... metalLayer.contentsScale = scale metalLayer.drawableSize = viewSize }

Oops, wrong reply field:

My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds.

In the view controller's view

 public var metalLayer = CAMetalLayer() 
override func viewDidAppear(_ animated: Bool) { ...
 view.layer.addSublayer(pipeline.metalLayer) } 

and manually replace drawableSizeWillChange

public func resize(_ viewSize: CGSize, _ scale: CGFloat) { ... 
metalLayer.contentsScale = scale 
metalLayer.drawableSize = viewSize } 

It doesn't support MTKView because MTKView is designed to provide a 2-dimensional view to project 3D graphics into.

Instead, you want to show a 3-dimensional view directly. You do this by using a new framework called Compositor Services to set up your metal rendering surface. To see how this works:

• Create a new visionOS app project in Xcode.
• In the project options under "Immersive Space Renderer", choose "Metal".
• Continue creating the project.

The code you're interested in is in the file <YourProjectName>App.swift and Renderer.swift. If you already understand Metal it should be pretty straightforward once you figure out the Compositor Services APIs.

Note that Metal apps don't support passthrough video at the moment, in case that's something you need.

Hope this helps.

MTKView uses CA/NSDisplayLink, but visionOS is driving the compositor. There's little point too, since you need the drawables from vision (side-by-side or layered).