What's the fastest way to render a RAW into a Metal texture?

Question

Using iOS 10 beta I want to capture a RAW and render it into a Metal texture.

This is what I am currently doing and it takes about 500ms using an iPhone 6S, I was hoping it would be faster:

// Create DNG from RAW
NSData* imageData = [AVCapturePhotoOutput DNGPhotoDataRepresentationForRawSampleBuffer:stillBuffer previewPhotoSampleBuffer:nil];

// Process DNG with no adjustments
CIFilter* rawFilter = [CIFilter filterWithImageData:imageData options:@{ (NSString*)kCGImageSourceTypeIdentifierHint : @"com.adobe.raw-image" }];
CIVector* nativesize = [rawFilter valueForKey:kCIOutputNativeSizeKey];
CIImage* result = rawFilter.outputImage;

// Create a Metal texture
id<MTLDevice> device = MTLCreateSystemDefaultDevice();
MTLTextureDescriptor* td = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatRGBA16Unorm width:nativesize.X height:nativesize.Y mipmapped:NO];
td.usage = MTLTextureUsageShaderWrite;
id<MTLTexture> texture = [device newTextureWithDescriptor:td];

// Render result into Metal texture
CIContext* context = [CIContext contextWithMTLDevice:device options:@{ kCIContextWorkingFormat : [NSNumber numberWithInteger:kCIFormatRGBAh] }];
[context render:result toMTLTexture:texture commandBuffer:nil bounds:(CGRect){ 0, 0, nativesize.Y, nativesize.X } colorSpace:CGColorSpaceCreateWithName( kCGColorSpaceLinearSRGB )];

Is this the best way to do it?

AVFoundation

1.4k

Posted by

Trickson

Reply

Add a Comment

Answer 1

Use the following approach instead:

CMSampleBufferRef rawSampleBuffer; // from your AVCapturePhotoCaptureDelegate callback
NSDictionary* rawImageAttachments = (__bridge_transfer NSDictionary *)CMCopyDictionaryOfAttachments(kCFAllocatorDefault, rawSampleBuffer, kCMAttachmentMode_ShouldPropagate);
CIContext* context = [CIContext contextWithMTLDevice:[...your device...]];
id<MTLTexture> texture = [... initialize your textue... ]
CIFilter* rawFilter = [CIFilter filterWithCVPixelBuffer:CMSampleBufferGetImageBuffer(rawSampleBuffer) properties:rawImageAttachments options:[... your options ...]];
[context render:rawFilter.outputImage toMTLTexture:texture commandBuffer:[...] bounds:[...] colorSpace:[...]]

Posted by

thomas.pintaric

Add a Comment

What's the fastest way to render a RAW into a Metal texture?

Replies