Which Camera API to use within iMessage Extension

I would like to present a live camera view within the MessagesViewController. There are quite a few different camera APIs to choose from, for live preview I picked the below code which works fine in a standard app. (Tested on iOS 10 as well.)


Which API should I be using to present a live camera preview?


I have added the plist item for camera permission message.


Thanks in advance!



-(void)addLiveCamera:(UIView*)camView {
    self.cameraCaptureSession = [AVCaptureSession new];
    NSArray *availableCameraDevices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
    AVCaptureDevice *backCameraDevice=nil,*frontCameraDevice=nil;
    for( AVCaptureDevice *device in availableCameraDevices ) {
        if( device.position == AVCaptureDevicePositionBack ) {
            backCameraDevice = device;
        } else if ( device.position == AVCaptureDevicePositionFront ) {
            frontCameraDevice = device;
        }
    }
    NSError *error = nil;
    AVCaptureDeviceInput *cameraInput = [AVCaptureDeviceInput deviceInputWithDevice:frontCameraDevice error:&error];
    if( [self.cameraCaptureSession canAddInput:cameraInput] ) {
        [self.cameraCaptureSession addInput:cameraInput];
        AVCaptureVideoPreviewLayer *previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:self.cameraCaptureSession];
        previewLayer.videoGravity = AVLayerVideoGravityResize;
        previewLayer.frame = camView.bounds;
        [camView.layer addSublayer:previewLayer];
        camView.backgroundColor = [UIColor clearColor];
        [self.cameraCaptureSession startRunning];
    }
}

Accepted Reply

It does appear that the camera capture isn't working correctly in Seed 1. This issue should be resolved in Seed 2.

Replies

Are you first requesting authorization to use the camera? I'm not sure what the correct API would be for what you want, but you'll need to get user authorization to use the camera first. If the API you're trying to use isn't working in an iMessage app extension, that would be worth getting written up in a bug report.

I am requesting authorization via the plist setting - NSCameraUsageDescription "message explaining why use camera"


When I step through the code, no errors are detected and the "startRunning" method does get called. In another stand-alone app I use the exact same code and it works fine, the UIView specified has a live view of the camera.


During the State of the Platform keynote, it was mentioned that the Messages App SDK would have access to the camera.


Some sample code would go a long towards helping out in this case! :-)

Having the NSCameraUsageDescription entry in the Info.plist does not actually request access to the camera. You need to use the AVCaptureDevice APIs to request authorization to use the camera before you do. Are you doing that?

Yes, in "viewDidLoad" on the view controller... the actual use currently comes after pressing a UIButton that calls up the preview layer code (above)


To clarify, I moved the AVCaptureSession code from the above code into this completionHandler block


    [AVCaptureDevice requestAccessForMediaType:AVMediaTypeVideo completionHandler:^(BOOL granted) {
        // Will get here on both iOS 7 & 8 even though camera permissions weren't required
        // until iOS 8. So for iOS 7 permission will always be granted.
        if (granted) {
            // Permission has been granted. Use dispatch_async for any UI updating
            // code because this block may be executed in a thread.
            dispatch_async(dispatch_get_main_queue(), ^{
                self.cameraCaptureSession = [AVCaptureSession new];
                NSArray *availableCameraDevices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
                for( AVCaptureDevice *device in availableCameraDevices ) {
                    if( device.position == AVCaptureDevicePositionBack ) {
                        self.backCameraDevice = device;
                    } else if ( device.position == AVCaptureDevicePositionFront ) {
                        self.frontCameraDevice = device;
                    }
                }
            });              
        } else {
            // Permission has been denied.
        }
    }];

I am experiencing the same issues getting a live camera view inside an iMessage extension.


I'm using the CameraViewController.swift file from the updated AVCam project. This works fine in the standalone app, but when used inside the extension after the session is started I immediately receive an AVCaptureSessionWasInterruptedNotification with a value of AVCaptureSessionInterruptionReason.videoDeviceNotAvailableInBackground


All permissions (camera / microphone / photolibrary) are requested and given, so I'm wondering if I am missing something on my part or should file a radar?

I filed a bug (26957474) with a sample project.


You may be on to something with the AVCaptureSessionInterruptionReason.videoDeviceNotAvailableInBackground notification... how does a Messages App assert focus, is that possible??

Thanks for reporting the bug. I'm experiencing the same issue. Will Apple notify you when it's resolved or do I just need to check future iOS 10 releases? I'm new to programming for Beta iOS.

Either is possible. I will report if I hear anything! :-)

It does appear that the camera capture isn't working correctly in Seed 1. This issue should be resolved in Seed 2.

I can confirm that camera API usage as outlined above does indeed work in beta 2. Yay! 🙂

It's not working for me in beta 2. What did you have to add to info.plist? I must be missing something.


Thanks

Dave

<key>NSCameraUsageDescription</key>

<string>description of camera use goes here</string>

Thanks!

Did you messages text field cover your camera controller when presented?

Does this mean that camera and microphone access is allowed in an iMessage Extension? This current documentation for app extensions states that access to the camera or microphone are now allowed within extension.
https://developer.apple.com/library/prerelease/content/documentation/General/Conceptual/ExtensibilityPG/ExtensionOverview.html#//apple_ref/doc/uid/TP40014214-CH2-SW6