7 Replies
      Latest reply on Oct 1, 2018 6:15 PM by rainhut
      xaphod Level 1 Level 1 (0 points)

        I'm testing out the new iOS 12 AVCapturePhotoOutput.portraitEffectsMatte.

        When it works, it's really great!

        But for some reason the matte doesn't get generated if the phone is standing up in a portrait orientation (the matte is always absent no matter what lighting conditions I tried). In the same conditions, the matte IS generated if I change the orientation of the phone such that it is in landscape orientation.

         

        Is this a known limitation of portraitEffectsMatte, or am I missing something?

         

        thanks for any help

        -tim

        • Re: iOS 12 .portraitEffectsMatte only works if device in landscape orientation?
          bford Apple Staff Apple Staff (315 points)

          Portrait effects matte generation is not dependent on orientation. It _is_ dependent on a person being in the scene. If there are no detected faces in the frame, the portrait effects matte will not be generated. This high res foreground background segmentation is currently trained on people, so if there are no people in the frame, it cannot be generated.

            • Re: iOS 12 .portraitEffectsMatte only works if device in landscape orientation?
              xaphod Level 1 Level 1 (0 points)

              Hi bford,

              There were always people in the frame.

              The portrait matte is simply not generated in potrait orientation even if there are people present, and the exact same people present in landscape orientation makes it work.

              One thing I had an idea about: I had hard-set the AVCaptureConnection.videoOrientation to .landscapeRight. Perhaps the portraitMatte generation requires the image to be in .up configuration to find the faces etc? To test this i've started to set .videoOrientation as the device rotates.

              I don't have the device anymore to try this out, but I will soon.

                • Re: iOS 12 .portraitEffectsMatte only works if device in landscape orientation?
                  rainhut Level 1 Level 1 (0 points)

                  There’s not much documentation on this. I’m able to save a portrait photo with a matte, but the matte is in a different orientation than the image. The odd thing is that the portrait image works with Apple‘s own camera App, so it must adjust the Matt to match the image.

                   

                  I’ve been trying to figure out how to use the AVCapturePhotoFileDataRepresentationCustomizer but no luck.

                    • Re: iOS 12 .portraitEffectsMatte only works if device in landscape orientation?
                      bford Apple Staff Apple Staff (315 points)

                      Closing the loop here. Thanks for reporting the issue. A bug was discovered in the -[AVPortraitEffectsMatte portraitEffectsMatteByApplyingExifOrientation:] when passing the native orientation of the buffer. So on the DualCamera, if you request a portraitEffectsMatteByApplyingExifOrientation with the equivalent of .landscapeRight, it will give you a nil buffer.

                       

                      The workaround is to not try to make a derivative portrait effects matte when no rotation / mirroring is necessary.

                        • Re: iOS 12 .portraitEffectsMatte only works if device in landscape orientation?
                          rainhut Level 1 Level 1 (0 points)

                          Portrait photos taken with the default camera have orientation matching images and mattes. But when taken with the APIs like below, the matte is rotated 90 degrees left of the image. So if we cannot currently use ApplyExifOrientation, how do we take photos that require no rotation like the default camera?

                           

                          From didFinishProcessingPhoto

                           

                                     PHPhotoLibrary.requestAuthorization { status in

                                          guard status == .authorized else { return }

                                          PHPhotoLibrary.shared().performChanges(

                                              {

                                                  let creationRequest = PHAssetCreationRequest.forAsset()

                                                  let options = PHAssetResourceCreationOptions()

                                               

                                                  //options.shouldMoveFile = true

                                                  creationRequest.addResource(with: .photo, data: photo.fileDataRepresentation()!, options:options)

                                          }, completionHandler: nil)

                                      }

                           

                                  }