Can textures be used with GPU driven indirect command buffers in Metal on A11?

Is it possible to use textures with GPU-driven indirect command buffers on Metal running on the A11 hardware?


First, the Metal indirect command buffer API does not have a `set_texture` method, just `set_vertex_buffer` and `set_fragment_buffer`. However, with argument buffers, a texture can be placed inside a buffer, so in theory it should be possible to use textures in indirect command buffers. Whenever I try this though, the MTLCompiler crashes.


Is this a bug, or is it an intended feature that textures can't be used in indirect command buffers? If so, that's a major limitation, and I'm a little surprised that it isn't called out anywhere that I can find in the documentation. On the other hand, Apple's only code sample of indirect command buffers doesn't use any textures, so I have my suspicions.


Here's some sample code to demonstrate the MTLCompiler crash. If you uncomment the `set_fragment_buffer` line that sets the argument buffer containing textures, MTLCompiler crashes with "MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED"


```

struct ICBContainer {

command_buffer commandBuffer [[ id(0) ]];

};


typedef struct TextureData {

texture2d<half> albedo [[ id(1) ]];

} TextureData;


kernel void encodeDraw(uint nodeId [[ thread_position_in_grid ]],

device VertexIn *vertices [[ buffer(BufferIndexVertices) ]],

device uint *indices [[ buffer(BufferIndexIndices) ]],

device ICBContainer *container [[ buffer( BufferIndexICBContainer ) ]],

device TextureData *textureData [[ buffer(BufferIndexTextureData) ]],

device InstanceData* instances [[ buffer( BufferIndexInstances ) ]],

constant VoxelUniforms *uniforms [[ buffer(BufferIndexVoxelUniforms)]], //nb has to be passed by pointer in order to be encoded

device ObjectParameters *parameters [[ buffer(BufferIndexObjectParameters) ]]

) {



InstanceData instance = instances[nodeId];

float4x4 modelMatrix = instance.modelMatrix;

// TODO frustum culling


instances[nodeId].normalViewMatrix = rotationMatrix(uniforms->viewMatrix) * rotationMatrix(modelMatrix);

instances[nodeId].shadowMVPTransformMatrix = clipToTexture * uniforms->shadowViewProjectionMatrix * modelMatrix;


render_command cmd(container->commandBuffer, nodeId);


cmd.set_vertex_buffer(instances, BufferIndexInstances);

cmd.set_vertex_buffer(vertices, BufferIndexVertices);

cmd.set_vertex_buffer(uniforms, BufferIndexVoxelUniforms);

cmd.set_fragment_buffer(uniforms, BufferIndexVoxelUniforms);

// cmd.set_fragment_buffer(textureData, BufferIndexTextureData); // uncomment this and compilation crashes


cmd.draw_indexed_primitives(primitive_type::triangle,

parameters[instance.meshId].indexCount,

indices, 1, parameters[instance.meshId].baseVertex,

nodeId);

};

```

Up vote post of Utsira Down vote post of Utsira
1.9k views
Posted by
Utsira
Reply
Add a Comment

Replies

I'm running into the same limitation + compile error with my attempt, and am curious if you ever found a work around?


Like you, I suspect that this might just be a (current?) limitation of indirect command buffers combined with argument buffers, and agree it's suspicious that Apple's samples never use textures.


I suspect the root cause of the compile failure is that argument buffers around bound to a specific MTLFunction (e.g. they are created with newArgumentEncoderWithBufferIndex:) and so it seems problematic to "pass through" an arugment buffer from a kernel shader to a vertex or fragment shader, as it would be created specifically for a shader at a given buffer index, and then couldn't be bound to the compute kernel (or vice versa). The fact that the failure is in runtime compile seems like they just didn't handle this case, though? I would have expected a build-time failure or assert, or at least a Metal Validation failure.


That said, I'm perplexed by the lack of "set_vertex_texture" or "set_fragment_texture" in the GPU render_command API, as if we can't set argument buffers of textures, and also can't set textures directly, I'm unclear as to how we get textures set on the vertex or fragment shaders invoked by an indirect command buffer render_command.draw_* call.


Perhaps the route is via the MTLIndirectCommandBufferDescriptor inheritBuffers property? That is, setting inheritBuffers to true, then setting textures on the CPU in the MTLRenderCommandEncoder prior to invoking the indirect command buffer? I'm not sure if that would then allow "overriding" of the vertex and fragment buffers via the indirect command buffer on the GPU, though, and also seems to greatly limit the flexibility of indirect command buffers even if if works.


Thoughts?

Posted by
GenerationLoss
Up vote reply of GenerationLoss Down vote reply of GenerationLoss
Add a Comment

Well, sadly, this also doesn't seem to work.

  • I tried setting inheritBuffers to true on the MTLIndirectCommandBufferDescriptor
    • This results in an immediate error of "Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4)".
    • I'm not really sure what "inherit buffers" is supposed to do. I guess its for if you set zero buffers in our indirect command buffer on the GPU, and simply issue draw_* commands in it? This seems of questionable value, as it would almost degenerate to a (very complex) instanced rendering, albeit with the option to do some more advanced dispatch logic on the GPU.
  • I tried leaving inheritBuffers as false, but still setting the argument buffer of textures prepared for the fragment shader directly on my MTLRenderCommandEncoder prior to calling executeCommandsInBuffer (which I called on the same MTLRenderCommandEncoder). I was hoping that buffers that weren't set by the indirect command buffer would be inherited regardless of the "inheritBuffers" flag

    Again, immediate "Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4)"

  • I tried directly putting textures / samplers as direct argument to my fragment shader (e.g. no argument buffer) and trying to sample them I get the error of "Fragment shader cannot be used with indirect command buffers" when creating the MTLRenderPipelineState
    • It seems like the error really means "textures cannot be used directly on shaders with indirect command buffers"
    • An equivalent error occurs when trying with vertex shaders


I guess at this point I'm completely stumped as to how to set textures for use with an indirect command buffer:

  • You can't seem to put them in an argument buffer to the compute kernel and pass them through with set_vertex_buffer() / set_fragment_buffer(), as you get the runtime compile error
  • You can't inherit them from within an argument buffer that you set on the encoder on the CPU, regardless of the value of "inherit buffers", as you get GPU errors in either case
  • You can't directly put them as arguments of the vertex or fragment shader, or the MTLRenderPipline creation will fail


Have you had any success?

Posted by
GenerationLoss
Up vote reply of GenerationLoss Down vote reply of GenerationLoss
Add a Comment