Strange Metal Artefacts

In a simple Metal app that renders a grid of 2D Textures rotated and scaled I get the following strange artefacts (only on a MacBook Pro - on an iPad Pro the same code works perfectly). At some small regions on the screen (mainly little squares of ca. 10x10 pixels) the Fragment shader accesses the wrong texture, that is, NOT the one set by setFragmentTexture:atIndex:. These regions depend on the texture, are much smaller than the drawing primitive (the drawn triangle), and are on a fixed position on the screen, that is, they do not depend on the position in the texture. They remain on the same position, even if all other buffers (Vertex-Positions, Uniforms, etc.) are reloaded and changed, as long as at the particular screen position the same texture is displayed.


I am just switching from OpenGL to Metal. It could well be that I am doing something the wrong way. I have no idea what kind of mistake could provoke such a strange behavior... Could it be a hardware problem? Any ideas? (The same code works perfectly on an iPad though, with the only difference that MTLStorageModeShared is used for the textures and accordingly replaceRegion:... is not called)


Screenshots: https://nc.fhoermann.org/index.php/s/aM86J4g5LXHQYRZ

(The black regions are flickering all the time - I included two different outputs for the same encoding.)


I use a MacBook Pro (15 ", 2019), 2,3 GHz Intel Core i9, 16 GB 2400 MHz DDR4,

Intel UHD Graphics 630 1536 MB and Radeon Pro Vega 20 4 GB

macOS 10.14.5


Here are the relevant code fragments:

Initialization:


- (void)_loadMetalWithView:(nonnull MTKView *)view;
{
    view.depthStencilPixelFormat = MTLPixelFormatInvalid;
    view.colorPixelFormat = MTLPixelFormatBGRA8Unorm;
    view.sampleCount = 1;

    _mtlVertexDescriptor = [[MTLVertexDescriptor alloc] init];

    _mtlVertexDescriptor.attributes[0].format = MTLVertexFormatFloat2;
    _mtlVertexDescriptor.attributes[0].offset = 0;
    _mtlVertexDescriptor.attributes[0].bufferIndex = 0;

    _mtlVertexDescriptor.attributes[1].format = MTLVertexFormatFloat2;
    _mtlVertexDescriptor.attributes[1].offset = 0;
    _mtlVertexDescriptor.attributes[1].bufferIndex = 1;

    _mtlVertexDescriptor.layouts[0].stride = 8;
    _mtlVertexDescriptor.layouts[0].stepRate = 1;
    _mtlVertexDescriptor.layouts[0].stepFunction = MTLVertexStepFunctionPerVertex;

    _mtlVertexDescriptor.layouts[1].stride = 8;
    _mtlVertexDescriptor.layouts[1].stepRate = 1;
    _mtlVertexDescriptor.layouts[1].stepFunction = MTLVertexStepFunctionPerVertex;

    id<MTLLibrary> defaultLibrary = [_device newDefaultLibrary];

    id <MTLFunction> vertexFunction = [defaultLibrary newFunctionWithName:@"vertexShader"];
    id <MTLFunction> fragmentFunction = [defaultLibrary newFunctionWithName:@"fragmentShader"];

    MTLRenderPipelineDescriptor *pipelineStateDescriptor = [[MTLRenderPipelineDescriptor alloc] init];
    pipelineStateDescriptor.label = @"MyPipeline";
    pipelineStateDescriptor.sampleCount = view.sampleCount;
    pipelineStateDescriptor.vertexFunction = vertexFunction;
    pipelineStateDescriptor.fragmentFunction = fragmentFunction;
    pipelineStateDescriptor.vertexDescriptor = _mtlVertexDescriptor;
    pipelineStateDescriptor.colorAttachments[0].pixelFormat = view.colorPixelFormat;

    NSError *error = NULL;
    _pipelineState = [_device newRenderPipelineStateWithDescriptor:pipelineStateDescriptor error:&error];
    if (!_pipelineState)
    {
        NSLog(@"Failed to created pipeline state, error %@", error);
    }

    _commandQueue = [_device newCommandQueue];
    
    MTLTextureDescriptor *textureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatR32Float
                                                                                                 width:128
                                                                                                height:128                                                                                    
                                                                                             mipmapped:NO];
    textureDescriptor.storageMode = MTLStorageModeManaged;
    textureDescriptor.usage = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite;    
    _Textures = [[NSMutableArray alloc] init];
    
    for(int i=0;i<NMEMTEXTURE;i++)
    {
       [_Textures addObject:[_device newTextureWithDescriptor:textureDescriptor]];
    }   
}   


Upload of the textures:


   [_Textures[texture] replaceRegion:MTLRegionMake2D(0,0,128,128)
      mipmapLevel:0
      withBytes:pixels
    bytesPerRow:(NSUInteger)(128*sizeof(float))];


Drawing Code:


- (void)drawInMTKView:(nonnull MTKView *)view
{
    dispatch_semaphore_wait(_inFlightSemaphore, DISPATCH_TIME_FOREVER);

    id <MTLCommandBuffer> commandBuffer = [_commandQueue commandBuffer];
    commandBuffer.label = @"MyCommand";

    __block dispatch_semaphore_t block_sema = _inFlightSemaphore;
    [commandBuffer addCompletedHandler:^(id<MTLCommandBuffer> buffer)
     {
         dispatch_semaphore_signal(block_sema);
     }];
    
    MTLRenderPassDescriptor* renderPassDescriptor = view.currentRenderPassDescriptor;

    if(renderPassDescriptor != nil) {

        id <MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor];
        
        renderEncoder.label = @"MyRenderEncoder";

        [renderEncoder setRenderPipelineState:_pipelineState];

    [renderEncoder setVertexBuffer:_dynamicUniformBuffer
                            offset:_uniformBufferOffset
                           atIndex:2];

    [renderEncoder setFragmentBuffer:_dynamicUniformBuffer
                              offset:_uniformBufferOffset
                             atIndex:2];
    
     [renderEncoder setVertexBuffer:_VertexPositions[list]
                            offset:0
                           atIndex:0];

     [renderEncoder setVertexBuffer:_VertexGenerics[list]
                            offset:0
                           atIndex:1];
    
    for(int i=0;i<vp->idrawrect[list];i++)
    {
        DRAWRECT *dr = &vp->drawrect[list][i];
        [renderEncoder setFragmentTexture:_Renderer->_Textures[dr->texture[0]] atIndex:0];
        [renderEncoder drawPrimitives:MTLPrimitiveTypeTriangleStrip
                            vertexStart:i*4
                            vertexCount:4];
    }


        [renderEncoder endEncoding];
        [commandBuffer presentDrawable:view.currentDrawable];
    }
    
    [commandBuffer commit];
}


Shader:


typedef struct
{
    matrix_float4x4 projectionMatrix;
    matrix_float4x4 modelViewMatrix;
    float params[6];
} Uniforms;

typedef struct
{
    float2 position [[attribute(0)]];
    float2 texCoord [[attribute(1)]];
} Vertex;

typedef struct
{
    float4 position [[position]];
    float2 texCoord;
} ColorInOut;

vertex ColorInOut vertexShader(Vertex in [[stage_in]],
                               constant Uniforms & uniforms [[ buffer(2) ]])
{
    ColorInOut out;

    float4 position = float4(in.position, 0.0, 1.0);
    out.position = uniforms.projectionMatrix * uniforms.modelViewMatrix * position;
    out.texCoord = in.texCoord;

    return out;
}

fragment float4 fragmentShader(ColorInOut in [[stage_in]],
                               constant Uniforms & uniforms [[ buffer(2) ]],
                               texture2d<float> colorMap     [[ texture(0) ]])
{
    constexpr sampler colorSampler(mip_filter::none,
                                   mag_filter::nearest,
                                   min_filter::nearest);
     int iter;

        float rr = 0.5*colorMap.sample(colorSampler, in.texCoord.xy).r;
        
        if(rr>=0.0)
            iter = int(rr*2147483648.0);
        else
            iter = int(-rr*2147483648.0);

        float i = float(iter)+0.5;
        
        float4 c = float4(0.5*(1.0+sin(i*uniforms.params[0]+uniforms.params[1])),
                          0.5*(1.0+sin(i*uniforms.params[2]+uniforms.params[3])),
                          0.5*(1.0+sin(i*uniforms.params[4]+uniforms.params[5])), 1.0);
    
    return c;
}


Thanks a lot!

Replies

Check the documentation for

replaceRegion:mipmapLevel:withBytes:bytesPerRow:


It doesn't synchronise. What might be happening: you're submitting the draw call to the GPU, then updating the texture with replaceRegion, and it's actually uploading the texture while the shader from the last draw call is rendering with it. If so, just adding a waitUntilCompleted should fix it.

Thanks a lot for your suggestion. I tried this already before and also rendered the scene with ca. 1fps waiting half a second after and before the calls to replaceRegion: to make sure that it is not a problem related to syncronization. The same effect occurs.


(- of course the textures used for replaceRegion were supposed to be old ones anyway, not being in a render pipeline for a significant time - the test was to make sure that there is no mistake in the part of the program organizing this)

In that case, I'd do a frame capture and take a look to see where the problem is (should be quite easy to find then, if capturing doesn't 'fix' the issue - if it does, it's likely sync related)

I tried the capure and indeed the artefacts are still there and change from frame to frame in the Xcode-Window displaying the colorAttachment (that is, they change for the same capture each time Xcode redisplays the output). The debugging however does not work; not even for the sample project: "https://developer.apple.com/documentation/metal/migrating_opengl_code_to_metal?language=objc". The error message is:


Unable to create shader debug session

Error DYPShaderDebuggerErrorDomain:2:

Error creating instrumented render pipeline state: Error Domain=CompilerError Code=1 "Link failed: fragment shader is reading the render_target_array_index but the vertex shader does not write it" UserInfo={NSLocalizedDescription=Link failed: fragment shader is reading the render_target_array_index but the vertex shader does not write it}


Of course there is no "render_target_array_index" specified anywhere in the Shader.


I have no idea what's happening, nor whether this is related to the problem we are discussing.

It might be that it treats the render target as an array even if there's only one, in which case you're still using it even though you might not specify it in the code. It certainly looks like the VS writes the vertex data out though. Odd. Are you using the latest Xcode?


The only other issue I can see is in your fragment shader, line 40, you refer to the vertex as 'n' instead of 'in', which could cause that error but I'm pretty sure it wouldn't compile, so that's probably just an accident while posting.

Yes, I did use the latest version of Xcode, but actually an update to 10.15 Catalina resolved the problem with the debugger.

Now the Capture and Debugging works, but at the pixels within the Artefact the debugger shows a different return value for the fragment shader (the correct one!) than the (wrong) value in the output. See new screenshot at https://nc.fhoermann.org/index.php/s/aM86J4g5LXHQYRZ

Even debugging several different pixels (within the artefact) several times I was not able to produce the error in the debugger.


PS. Sorry, I introduced typos in the shader code posting it, I'll try to edit that.