Copying depth texture to buffer only works for large sizes

Question

Athenstean OP

Created Sep ’16

Replies 6

Boosts 0

Views 3k

Participants 3

Hi all,

I'm trying to do a "render to texture" (i.e. create a render pass that is separate from the main rendering code, separate textures, etc) and read back the depth texture and store it in an image in my application. This is suposed to be a one-time step when loading a new scene. I had found hints on stackoverflow that one has to use MTLBlitCommandEncoder copyFromTexture because with recent releases the texture is in private memory. So far so good. My problem is that this only seems to work for large sizes, if the render resolution is for example 512x512 (or smaller) the copied value in the buffer looks partial or empty as if the blit would have occured in the middle of rendering (2 of 10 drawables not shown or in a larger mesh there are gaps that look like triangles haven'be been rasterized). If I keep the code exactly the same and increase the render size to 2048x2048 it works perfectly. Also if I use the same code to copy the color attachment it works all the time. Finally, I used instrument to verify that yes, first the render encoder is done and then the blit encoder.

The interesting part is: If I artificially call this code at the beginning of every frame and capture a frame in XCode, it says: "Your application created a command encoder but did not encode any work on it" for the line that creates the encoder although just two lines later in the frame capture there indeed is the copyFromTexture call.

Anyone got an idea why this might happen? Any suggestion is much appreciated. Thanks

Alex

P.S.: I'm running iOS 9.3.5 on an iPhone 6 and an iPad Pro. Implementation roughly looks like this (using non multi-sampling):

MTLRenderPassDescriptor * renderPass = [MTLRenderPassDescriptor renderPassDescriptor];
MTLTextureDescriptor * colorBufferDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatBGRA8Unorm width:imageSize.getWidth() height:imageSize.getHeight() mipmapped:NO];
colorBufferDescriptor.usage = MTLTextureUsageRenderTarget;
renderPass.colorAttachments[0].texture = [self.mtlDevice newTextureWithDescriptor:colorBufferDescriptor];
renderPass.colorAttachments[0].clearColor = MTLClearColorMake(0.0, 0.0, 0.0, 0.0);
renderPass.colorAttachments[0].loadAction = MTLLoadActionClear;
           
MTLTextureDescriptor * depthBufferDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatDepth32Float_Stencil8 width:imageSize.getWidth() height:imageSize.getHeight() mipmapped:NO];
depthBufferDescriptor.usage = MTLTextureUsageRenderTarget;
renderPass.depthAttachment.texture = [self.mtlDevice newTextureWithDescriptor:depthBufferDescriptor];
renderPass.depthAttachment.loadAction = MTLLoadActionClear;
renderPass.stencilAttachment.texture = renderPass.depthAttachment.texture;

id <MTLCommandBuffer> commandBuffer = [self.mtlCommandQueue commandBuffer];
//
// [...] <- doing render encoding here
//
id<MTLBuffer> depthImageBuffer = [self.mtlDevice newBufferWithLength:(4 * pixelCount) options:MTLResourceOptionCPUCacheModeDefault];
id<MTLBlitCommandEncoder> blitCommandEncoder = commandBuffer.blitCommandEncoder;
blitCommandEncoder.label = @"Depth buffer to CPU blit";
[blitCommandEncoder copyFromTexture:renderPass.depthAttachment.texture
                        sourceSlice:0 sourceLevel:0 sourceOrigin:MTLOriginMake(0, 0, 0)
                         sourceSize:MTLSizeMake(imageSize.getWidth(), imageSize.getHeight(), 1)
                           toBuffer:depthImageBuffer destinationOffset:0
             destinationBytesPerRow:(4 * imageSize.getWidth())
           destinationBytesPerImage:(4 * pixelCount) options:MTLBlitOptionDepthFromDepthStencil];

[blitCommandEncoder endEncoding];

[commandBuffer commit];
[commandBuffer waitUntilCompleted];

Answered by MikeAlpha in 177574022

Metal programming guide has this to say about depth texture store action: " For depth and stencil attachments,

MTLStoreActionDontCare

is the default store action, because those attachments typically do not need to be preserved after the rendering pass is complete.". And your code doesn't seem to set store action for depth texture...

Boost

Answer 1

MikeAlpha OP

Sep ’16

From my experience there are many problems with blit command encoder on iOS (iPad pro). Mipmap generation does not work, buffer clearing works only once (second usage crashes application), copying beetween textures and buffers cayses problems and so on. No, I haven't filled rdars for this (yet).

In my application (macOS/iOS) there is MTLUtility static class which has most of blit functions implemented. On macOS these are actual blit command encoder calls. On iOS, every single function is implemented as either rendering to texture or compute kernel.

IN your case this should be easy to implement as render to texture.

regards

MIchal

0

Answer 2

Athenstean OP

Sep ’16

Thanks a lot! I hadn't thought about that option... actually haven't written a compute shader before. Something like this should do the job, right?

#include <metal_stdlib>
using namespace metal;

kernel void filter_main(
          depth2d<float, access::read>   inputImage   [[ texture(0) ]],
          texture2d<float, access::write>  outputImage  [[ texture(1) ]],
          uint2 gid                                    [[ thread_position_in_grid ]]
          )
{
    float depthValue = inputImage.read(gid);
    outputImage.write(float4(depthValue, depthValue, depthValue, 1.0), gid);
}

Strangely enough it looks okay if I check the compute shader step in the frame capture list. But for some reason if I access the bytes (getBytes:bytesPerRow:fromRegion:mipmapLevel:) of the target texture it looks as if the input image was completely empty (=depthValue is always zero. If I output 1.0/depthValue/depthValue/1.0 it will actually produce a solid blue texture, so the shader is doing something). Tried inputImage.sample as an alternative but got same result.

0

Answer 3

MikeAlpha OP

Sep ’16

Code looks ok to me, especially with "blue testing" :-) I can think of few things to check. First is, maybe your depth buffer contains zeros? What about storeAction for depth texture? Have you looked at it in "frame capture"? AFAIR (not using z-buffers much) Xcode shows it nicely. Second is, from what I remember depth value (as saved in memory) and actual z coordinate (from vertices and used for depth testing) is not the same, and there is some kind of non-linear converting process beetween these two. If so, you might be reading proper values from depth buffer, but they aren't that far from zero and therefore not producing visible effects. I'd try something like:

clamp( abs( depthValue * 1000.0 ), 0.0, 1.0 ) as a rough check.

And finally, perhaps there is some problem with reading from depth texture (like Metal bug)?

hope that helps

MIchal

0

Answer 4

MikeAlpha OP

Sep ’16

Accepted Answer

Metal programming guide has this to say about depth texture store action: " For depth and stencil attachments,

MTLStoreActionDontCare

is the default store action, because those attachments typically do not need to be preserved after the rendering pass is complete.". And your code doesn't seem to set store action for depth texture...

0

Answer 5

Athenstean OP

Sep ’16

Oh man, you just made my day!!!! It works... actually both solutions now work! Once I set

renderPass.depthAttachment.storeAction = MTLStoreActionStore;

the compute shader worked perfectly so out of curiousity I switched back to the blit encoder code and it also works now! The confusing thing was that everything looked perfect in the frame capture but I guess in that scenario it copies over the temporary state of the textures. This also might explain why I ran into the problem only for "smaller" texture because large ones are probably allocated in a different area or something and therefore don't run that high a risk of the memory being recycled.

Thanks a lot for your help!

0

Answer 6

dcvz OP

Jun ’22

@Athenstean do you mind sharing how you achieved it with the compute shader instead?

0