How can I prevent AVCaptureVideoDataOutput from automatically downscaling image buffers?

Hi,

In our iOS App, we need to access uncompressed full-resolution frames from the back-facing video camera in an AVCaptureSession with an AVCaptureVideoDataOutput. Up to this point, we've been configuring our session using AVCaptureSessionPresetInputPriority (through the -setActiveFormat: API), but according to WWDC 2016 Session 501 "Advances in iOS Photography", RAW and Live Photo capture in iOS 10 (using AVCapturePhotoOutput) will only be supported for AVCaptureSessionPresetPhoto, which is why we're forced to make a switch to the -setSessionPreset: API.

Unfortunately, this change somehow causes AVCaptureVideoDataOutput to apply an internal downscaling operation (see example below) to the image buffers prior to delivering them to our AVCaptureVideoDataOutputSampleBufferDelegate callback! The delivered frame turns out to be much lower in resolution compared to the active device format and doesn't even match the format description provided by the AVCaptureVideoDataOutput's AVCaptureInputPort.

Code example (for iOS 9.3/XCode 7):

#import <UIKit/UIKit.h>
@import AVFoundation;

@interface ViewController : UIViewController<AVCaptureVideoDataOutputSampleBufferDelegate>
@property (nonatomic) AVCaptureSession* captureSession;
@property (nonatomic) AVCaptureVideoDataOutput* captureVideoDataOutput;
@property (nonatomic) AVCaptureDevice* videoDevice;
@property (nonatomic) dispatch_queue_t  sampleBufferQueue;
@end

@implementation ViewController

- (id)initWithCoder:(NSCoder *)aDecoder
{
    if (self = [super initWithCoder:aDecoder])
    {
        self.captureSession = [[AVCaptureSession alloc] init];
        self.sampleBufferQueue = dispatch_queue_create("sampleBufferQueue", DISPATCH_QUEUE_CONCURRENT);
    }
    return self;
}

- (void)configureCaptureSession;
{
    NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
    self.videoDevice = devices[[devices indexOfObjectPassingTest:^BOOL(AVCaptureDevice* device, NSUInteger idx, BOOL *stop) {
        if(device.position == AVCaptureDevicePositionBack) {
            *stop = TRUE; return(YES);
        }
        return(NO);
    }]];
    assert(self.videoDevice);

    [self.captureSession beginConfiguration];
    NSError* error;
    AVCaptureDeviceInput *videoDeviceInput = [AVCaptureDeviceInput deviceInputWithDevice:self.videoDevice error:&error];

    assert(videoDeviceInput && [self.captureSession canAddInput:videoDeviceInput]);
    [self.captureSession addInput:videoDeviceInput];

    self.captureSession.sessionPreset = AVCaptureSessionPresetPhoto;
    NSLog(@"videoDevice.activeFormat: %@", self.videoDevice.activeFormat);

    /
    self.captureVideoDataOutput = [[AVCaptureVideoDataOutput alloc] init];
    {
        assert([self.captureSession canAddOutput:self.captureVideoDataOutput]);
        [self.captureSession addOutput:self.captureVideoDataOutput];

        [self.captureVideoDataOutput setAlwaysDiscardsLateVideoFrames:YES];
        [self.captureVideoDataOutput setVideoSettings:@{ (__bridge NSString*)kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_420YpCbCr8BiPlanarFullRange)}];

        [self.captureVideoDataOutput setSampleBufferDelegate:self queue:self.sampleBufferQueue];
    }

    [self.captureSession commitConfiguration];
    NSLog(@"captureVideoDataOutput.videoSettings: %@", self.captureVideoDataOutput.videoSettings);
}
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    const size_t pixelBufferWidth = CVPixelBufferGetWidth(imageBuffer);
    const size_t pixelBufferHeight = CVPixelBufferGetHeight(imageBuffer);
    NSLog(@"Pixel buffer dimensions in delegate callback: %zu x %zu", pixelBufferWidth, pixelBufferHeight);

    for(AVCaptureInputPort* port in connection.inputPorts)
    {
        if(port.mediaType == AVMediaTypeVideo)
        {
            CMVideoDimensions videoDimensions = CMVideoFormatDescriptionGetDimensions(port.formatDescription);
            NSLog(@"AVCaptureInputPort video format dimensions: %d x %d", videoDimensions.width, videoDimensions.height);
            break;
        }
    }
}

- (void)viewDidDisappear:(BOOL)animated
{
    [self.captureSession stopRunning];
    [super viewDidDisappear:animated];
}

- (void)viewWillAppear:(BOOL)animated
{
    [super viewWillAppear:animated];
    [self.captureSession startRunning];
}

- (void) viewDidAppear:(BOOL)animated
{
    [super viewDidAppear:animated];
    [AVCaptureDevice requestAccessForMediaType:AVMediaTypeVideo completionHandler:nil];
}

- (void)viewDidLoad
{
    [super viewDidLoad];
    [self configureCaptureSession];
}

@end

Running the example above on an iPhone 6 (under iOS 9.3), the console output will read as follows (interesting parts marked bold):

--START--

videoDevice.activeFormat: <AVCaptureDeviceFormat: 0x12ee051f0 'vide'/'420f' 3264x2448, { 2- 30 fps}, fov:58.040, max zoom:153.00 (upscales @1.00), AF System:2, ISO:29.0-1856.0, SS:0.000025-0.500000>

captureVideoDataOutput.videoSettings: {

AVVideoScalingModeKey = AVVideoScalingModeResize;

Height = 750;

PixelFormatType = 875704422;

Width = 1000;

}

Pixel buffer dimensions in delegate callback: 1000 x 750

AVCaptureInputPort video format dimensions: 3264 x 2448

--END--

Note the discrepancies between the active device format (3264x2448) and the frame size inside the delegate callback (1000x750)!

Also note the undocumented dictionary keys AVVideoScalingModeKey, Width and Height in captureVideoDataOutput.videoSettings. (I tried using a combination of those keys as input to setVideoSettings:, but that only resulted in the error "- videoSettings dictionary contains one or more unsupported (ignored) keys:").

So, here are my questions:

• How can I prevent this unwanted downscaling from happening with AVCaptureSessionPresetPhoto?
• Will there be any way to support RAW and Live Photo capture (in iOS 10) trough the -setActiveFormat: API?