Sampler works when debugged from frame capture but not on MTKView

Hi! I'm trying to implement hard shadows on a custom Metal rendering engine. Here's what I'm currently doing:

  • First, an initial render pass to render the model from the sun's point of view and create a depth2D texture of the points of the model viewed from the sun's point of view (using an orthographic projection).
  • Secondly, another render pass, where for each point on the fragment shader I compute the equivalent point on the sun's frame of view, convert the coordinates to Metal's texture coordinate system, and sample the depth2D texture generated on the first pass using sample_compare to check if it's occluded or not in the sun's frame of reference. If it is occluded, I reduce the intensity of the output color, creating the shadow.

The problem: It seems like the texture is not being sampled properly. I've tried outputting the result of the sample_compare operation directly as the fragment color and it seems like it always returns 0. The final image is pure black.

The weird part: If I try to debug it using Xcode's GPU frame capture, the image is NOT black, and the shadows are casted properly. Here's a test on a single sphere, showing white if the fragment is lit and black if it's unlit:

And another test rendering a real model with full lighting. Note how on the actual view rendered on the app, all pixels in the fragment are seen as 'occluded' from the sun, so the whole model is in the shadows.

Why is the result of sample_compare non-zero on the GPU Frame capture but not on the actual image displayed?

Here's the relevant code of my second fragment (the one rendering the final image):

constexpr sampler shadowSampler(coord::normalized,
                                filter::linear,
                                mip_filter::none,
                                address::clamp_to_border,
                                border_color::opaque_white,
                                compare_func::less_equal);

float shadow_sample = shadowMap.sample_compare(shadowSampler,
                                               sphereShadowTextureCoord.xy,
                                               sphereShadowTextureCoord.z);

// According to the debugger, shadow_sample returns a value greater than 0 for the lit pixels and 0 otherwise. 
// Not sure it this is expected (I thought it would return 1.0 if lit, 0.0 if unlit).
float is_sunlit = 0;
if (shadow_sample > 0) {
    is_sunlit = 1;
}

// Output color
output.color = half4(shadedColor.r - 0.3 * (1 - is_sunlit), 
                     shadedColor.g - 0.3 * (1 - is_sunlit), 
                     shadedColor.b - 0.3 * (1 - is_sunlit), 
                     1.0);

Advice on what could be causing the problem (or how to debug it, since the debugging tools are not returning the expected results) would be appreciated. Thanks!

Answered by Andropov in 697807022

Fixed it! I was writing colorAttachments[1].storeAction = .store where I should have written depthAttachment.storeAction = .store. So the depth attachment was on the default mode for depth attachments, .dontCare. It worked on the GPU frame capture because it stores all textures, but not on the real app because it was being discarded before the next render pass.

Things I've tried so far:

  • Calling waitUntilCompleted() on the command encoder does not change anything.
  • Setting the shadow depth texture to compressed/non-compressed (using allowsGPUOptimizations) does not change anything.
  • I've checked that the shadow depth texture has a MTLPurgeableState of .nonVolatile so it's not being discarded.
  • Passing a fully 'white' (1.0) texture, which should always return true when sampled from the sample_compare with compare_func::less_equal. Same for a fully 'black' (0.0) texture, for what is worth...

Here are the shadow depth texture settings, anyway, in case any of it helps:

some : <CaptureMTLTexture: 0x600001b17e80> -> <AGXG13XFamilyTexture: 0x149e514c0>
    label = Shadow Depth Texture 
    textureType = MTLTextureType2D 
    pixelFormat = MTLPixelFormatDepth32Float 
    width = 1024 
    height = 1024 
    depth = 1 
    arrayLength = 1 
    mipmapLevelCount = 1 
    sampleCount = 1 
    cpuCacheMode = MTLCPUCacheModeDefaultCache 
    storageMode = MTLStorageModePrivate 
    hazardTrackingMode = MTLHazardTrackingModeTracked 
    resourceOptions = MTLResourceCPUCacheModeDefaultCache MTLResourceStorageModePrivate MTLResourceHazardTrackingModeTracked  
    usage = MTLTextureUsageShaderRead MTLTextureUsageRenderTarget 
    shareable = 0 
    framebufferOnly = 0 
    purgeableState = MTLPurgeableStateNonVolatile 
    swizzle = [MTLTextureSwizzleRed, MTLTextureSwizzleGreen, MTLTextureSwizzleBlue, MTLTextureSwizzleAlpha] 
    isCompressed = 1 
    parentTexture = <null> 
    parentRelativeLevel = 0 
    parentRelativeSlice = 0 
    buffer = <null> 
    bufferOffset = 0 
    bufferBytesPerRow = 0 
    iosurface = 0x0 
    iosurfacePlane = 0 
    allowGPUOptimizedContents = YES
    label = Shadow Depth Texture
    label = Shadow Depth Texture

Passing down a fully white texture makes the sampler work as expected on the GPU Frame Capture (the model is lit) but still keeps returning 0.0 on the real view (the whole model is unlit). Only exception, if the sample falls outside the texture, and the sampler is set to clamp_to_border with border_color::opaque_white.

If I set it to clamp_to_edge, for example, samples outside the texture return 0.0 (unlit) too. So it's as if the texture was fully black on the real view, for some reason...

Accepted Answer

Fixed it! I was writing colorAttachments[1].storeAction = .store where I should have written depthAttachment.storeAction = .store. So the depth attachment was on the default mode for depth attachments, .dontCare. It worked on the GPU frame capture because it stores all textures, but not on the real app because it was being discarded before the next render pass.

Sampler works when debugged from frame capture but not on MTKView
 
 
Q