MetalKit in SwiftUI

Hi: This thead was quite helpful with my own code, thank you!With the newly announced (WWDC 2023) metal shaders available in SwiftUI, I wonder how the standalone example that creates its own MTKView would be updated with the latest SwiftUI? It is not obvious, at least to me...

Supporting MetalKt in SwiftUI might make things easier, but even currently, rendering a MTKView with SwiftUI is no more or less involved than it is with AppKit. The code ends up structured a little differently, but it's the same code, and more or less the same methods.

I've implemented the same simple pixel shader roughly four different ways now - twice with AppKit, twice with SwiftUI. IMO, the best approach uses SwiftUI, and was derived from the example given above (posted a few years ago, by an anonymous Apple dev). The code turns out very similar either way, but AppKit also uses a storyboard (which makes it feel clunky and magical), and SwiftUI is the future etc, so I went with SwiftUI.

I'll include a minimal example here (for my own future reference, as much as anyone else's). The app's called Console. This is is the entrypoint, named ConsoleApp.swift:

import SwiftUI

@main
struct ConsoleApp: App {

    var body: some Scene { WindowGroup { MetalView() } }
}

#Preview { MetalView() }

The bulk of the Swift code is in a file named MetalView.swift. It initializes everything, including a buffer for passing vertices to the shader. These vertices are used by the vertex function to render a rectangle (formed from two triangles, in a triangle-strip), so the frag shader runs once for each pixel in the framebuffer.

import SwiftUI
import MetalKit

struct MetalView: NSViewRepresentable {

    static private let pixelFormat = MTLPixelFormat.bgra8Unorm_srgb
    static private let clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 0)

    class Coordinator: NSObject, MTKViewDelegate {

        var device: MTLDevice

        private var commandQueue: MTLCommandQueue
        private var pipelineState: MTLRenderPipelineState
        private var vertexBuffer: MTLBuffer

        override init() {

            device = MTLCreateSystemDefaultDevice()!

            let library = device.makeDefaultLibrary()!
            let descriptor = MTLRenderPipelineDescriptor()
            let vertices: [simd_float2] = [[-1, +1],  [+1, +1],  [-1, -1],  [+1, -1]]
            let verticesLength = 4 * MemoryLayout<simd_float2>.stride

            descriptor.label = "Pixel Shader"
            descriptor.vertexFunction = library.makeFunction(name: "init")
            descriptor.fragmentFunction = library.makeFunction(name: "draw")
            descriptor.colorAttachments[0].pixelFormat = MetalView.pixelFormat

            commandQueue = device.makeCommandQueue()!
            pipelineState = try! device.makeRenderPipelineState(descriptor: descriptor)
            vertexBuffer = device.makeBuffer(bytes: vertices, length: verticesLength, options: [])!

            super.init()
        }

        func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) { }

        func draw(in view: MTKView) {

            let buffer = commandQueue.makeCommandBuffer()!
            let descriptor = view.currentRenderPassDescriptor!
            let encoder = buffer.makeRenderCommandEncoder(descriptor: descriptor)!

            encoder.setRenderPipelineState(pipelineState)
            encoder.setVertexBuffer(self.vertexBuffer, offset: 0, index: 0)
            encoder.drawPrimitives(type: .triangleStrip, vertexStart: 0, vertexCount: 4)
            encoder.endEncoding()

            buffer.present(view.currentDrawable!)
            buffer.commit()
        }
    }

    // finish by defining the three methods that are required by `NSViewRepresentable` conformance...

    func makeCoordinator() -> Coordinator { Coordinator() }

    func makeNSView(context: NSViewRepresentableContext<MetalView>) -> MTKView {

        let view = MTKView()

        view.delegate = context.coordinator
        view.device = context.coordinator.device
        view.colorPixelFormat = MetalView.pixelFormat
        view.clearColor = MetalView.clearColor
        view.drawableSize = view.frame.size
        view.preferredFramesPerSecond = 60
        view.enableSetNeedsDisplay = false
        view.needsDisplay = true
        view.isPaused = false

        return view
    }

    func updateNSView(_ nsView: MTKView, context: NSViewRepresentableContext<MetalView>) { }
}

The shader boilerplate is simple. The vert function returns the vertices we pass into it, and the frag function just outputs purple, filling the viewport.

#include <metal_stdlib>

using namespace metal;

vertex float4 init(const device float2 * vertices[[buffer(0)]], const uint vid[[vertex_id]]) {

    return float4(vertices[vid], 0, 1);
}

fragment float4 draw() {

    return float4(0.5, 0, 0.5, 1);
}

In practice, you'll want to create more buffers (like the vertex buffer), and use them to pass data to the GPU (by pointer). You can easily populate the buffers with primitives (including SIMD types). Beyond that, you can create a bridging header, which allows you to define structs in a (separate) C header file, and instantiate them in Metal code and Swift. You can then put any data you want in the buffers.

Note: You should also look into using a semaphore to prevent race cases between the CPU and GPU threads.

Note: There are tutorials (mostly on Apple Developer and Medium) that cover all of these things in enough depth to figure the rest out from the official API docs.

One more thing: Looking through the docs, I noticed that the NSViewRepresentableContext<Self> type has an alias:

typealias Context = NSViewRepresentableContext<Self>

So, you can simplify the signatures for makeNSView and updateNSView. For example, this...

func updateNSView(_ nsView: MTKView, context: NSViewRepresentableContext<MetalView>) { }

...can be written like this:

func updateNSView(_ nsView: MTKView, context: Context) { }