Wanting to ditch MTKView.currentRenderPassDescriptor

Getting drawables directly from the CAMetalLayer is not compatible with MTKView becasue MTKView will not "know" that you've done this and cannot manage the framerate prperly. If you need just a drawable, call MTKView.currentDrawable. This, however, will not solve the issue you're running into as you git nil for this if you got nil for currentPassDescriptor.

I can't tell for sure what's going on with the info you've provided. Are you rendering to multiple views or CAMetalLayers with a single MTLCommandQueue? Have you presented all the drawables and commited the command buffers drawing to them when you see this stall?

I have 4

MTKViews

MTLCommandQueue

MTKView

NSView

NSWindow

CVDisplayLink

I do things this way with

CVDisplayLink

CVDisplayLink

MTKView

Is the maximum drawable count of 3 per

CAMetalLayer

CVDisplayLink

MTKView

dispatch_after

All works fairly well, but occasionally I receive the "nil"

currentPassDescriptor

Doing some CPU profiling, encoding the kernel shading is taking about 1.3ms, dispatching the Vertex/Fragment Shading (after obtaining the

currentPassDescriptor

Any suggestions would be helpful. I've been thoroughly stumped for some time.

BTW... this stall happens after I've done all my kernel shader work and in my first call to currentPassDescriptor -- just as I'm about to do my single pass of Vertex/Fragment shader work. After that work, I am presenting the drawable and commit the single command buffer.

The drawable count isn't per app. There is some drawable throttling done for each queue however. So if you you're using one queue to service multiple drawables there are some extra steps neccessary. That doesn't sound like the issue here.

Is there a reason you're using 4 separate CVDisplayLinks? This shouldn't be a problem in itself, but it means that you may be presenting frames at a different cadence which may force CoreAnimation and/or the macOS kernel to do some complicated resource tracking. I suggest spawning rendering for each frame from a single CVDisplayLink call back. You can encode work for each view with separate command buffers on separate threads, only the last thread should make the present calls.

To do this, instead of calling -[MTLCommandBuffer presentDrawable:] you would add a schedule handler to the last command buffer commited (of all the command buffers in your app). This would look something like this:

    id<MTLDrawable> drawable1 = mtkView1.currentDrawable;
    id<MTLDrawable> drawable2 = mtkView2.currentDrawable;
    id<MTLDrawable> drawable3 = mtkView3.currentDrawable;
    id<MTLDrawable> drawable4 = mtkView4.currentDrawable;
    [last_command_buffer addScheduledHandler:^(id commandBuffer)
     {
         [drawable1 present];
         [drawable2 present];
         [drawable3 present];
         [drawable4 present];
     }];
    
    
    [last_command_buffer commit];

Dan, this is incredibly helpful, yes.

The four MTKViews as subviews of one CAMetalLayer on a 4K monitor was a debugging configuration. In reality, these would end up on separate monitors on, say, MacPro6,1 hardware.

I just reconfigured things from MTKView/CAMetalLayer/NSWindow counts of 4/1/1 to 4/4/4 and all the hiccups disappeared.

The way I see things I was attempting to use 4 MTKViews with one set of 3 drawables on the single underlying CAMetalLayer.

BTW, the intent behind having 4 separate CVDisplayLinks was to attach each one to the refresh rate of the particular monitor via CVDisplayLinkCreateWithCGDisplay it was running on instead of using CVDisplayLinkCreateWithActiveCGDisplays.

Thanks again for the suggestion.

Okay. Using 4 display links makes sense if you each view will be on a different display. But you can't have a 4 views on multiple displays if they're in the same window. So in that case having 4 dsisplay links doens't make sense.

I'm not sure how you were using one set of drawable for 4 MTKViews. That doesn't seem possible, but bad things would happen if it were possible.

Thanks, Dan.

So, I take it that there's no means to have multiple MTKViews on a single background (NSView) with reliability -- ie. it won't stall on obtaining drawables. In short, I've solved the issue with the 4 main MTKViews by relegating them to their own window. This gave each MTKView its own set of drawables.

My 'downstream' requirement is this, however: while those MTKViews are playing their videos I want the main-screen UI to show mini regions of interest of the playing videos all within one window. I need a mouseable custom view in which I can move about in real time to pick a region of interest in the animating video frame. There are ≈4 such views (each subviews of an NSBox). The NSBox views are all subviews of a parent NSView. (One solution would be to create a floating window for each such view but that's not an ideal solution).

Below is my attempt to add a custom view with a backing CAMetalLayer instead. It also works about 95% of the time. I would appreciate any suggestions to get it right.

Previously this custom view was an MTKView but occasional drawable stalling occured (as previously).

I am now attempting to use a plain NSView for a custom view. I am adding this view as a subview of NSBox, just as before — and things are still mouseable and work most of the time — but 'drawables' stalling is still an issue.

If I use 'setNeedsDisplay:YES' on the view , the main thread brutally stalls when there's no drawable available, as expected. I can obviate by instead doing an async 'manualDraw' call on a background thread but still I end up with drawable starvation.

Below is my attempted solution using just a plain NSView.

In that custom view's initWithFrame: handler, I do this:

self.metalLayer = [[CAMetalLayer alloc] init];
if (!self.metalLayer)
    return nil;
[self.metalLayer setDevice:self.device];
[self.metalLayer setPixelFormat:MTLPixelFormatBGRA8Unorm];
[self.metalLayer setFramebufferOnly:YES];
[self.metalLayer setFrame:frameRect];
addedLayer=NO;  // can't add this CAMetalLayer to self.layer right now as self.layer is nil

Here are the other bits of that custom NSView. I've removed a lot of error-checking from my metalDraw method for the sake of brevity. I should mention that (1) the inputTexture is tripled-buffered from the upstream MTKView's command buffer and (2) the metal vertices mentioned here are only for a single quad, which corresponds the region of interest on the texture.

- (void)drawRect:(NSRect)dirtyRect {
    [super drawRect:dirtyRect];
    [self manualDraw];
}

-(BOOL)wantsUpdateLayer
{
    if (!addedLayer) {. // delayed addition of our self.metalLayer to self.layer
        if (self.layer) {
            addedLayer=YES;
            [self.layer addSublayer:self.metalLayer];
        }
    }
    return YES;
}

-(void)updateLayer {
    [self manualDraw];
}

- (void)manualDraw {
    @autoreleasepool {
        [self metalDraw];
    }
}

-(void)metalDraw {
  
    id<MTLCommandBuffer> commandBuffer = [metalCommandQueue commandBuffer];

    MTLRenderPassDescriptor *renderPassDescriptor = [MTLRenderPassDescriptor renderPassDescriptor];
    id<CAMetalDrawable> drawable = [self.metalLayer nextDrawable];
    renderPassDescriptor.colorAttachments[0].texture = drawable.texture;
    renderPassDescriptor.colorAttachments[0].loadAction = MTLLoadActionClear;
    renderPassDescriptor.colorAttachments[0].storeAction = MTLStoreActionStore;
    renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(1.0, 0.0, 0.0, 1.0);

    id<MTLRenderCommandEncoder> encoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor];
    MTLViewport viewPort = {0.0, 0.0, (double)self.drawableSize.width, (double)self.drawableSize.height, -1.0, 1.0};
    [encoder setViewport:viewPort];
    [encoder setRenderPipelineState:metalVertexPipelineState];
    NSUInteger vSize = _vertexInfo.metalVertexCount*sizeof(AAPLVertex);
    id<MTLBuffer> mBuff = [self.device newBufferWithBytes:_vertexInfo.metalVertices
                                                   length:vSize
                                                  options:MTLResourceStorageModeShared];
    [encoder setVertexBuffer:mBuff offset:0 atIndex:0];
    [encoder setVertexBytes:&_viewportSize length:sizeof(_viewportSize) atIndex:1];
    [encoder setFragmentTexture:inputTexture atIndex:0];
    [encoder drawPrimitives:MTLPrimitiveTypeTriangle vertexStart:0 vertexCount:_vertexInfo.metalVertexCount];
    [encoder endEncoding];

    [commandBuffer presentDrawable:drawable];
    [commandBuffer commit];
}