AVFoundation faceYaw metadata missing in iPhone 8 (Plus) only.

Hi,


I am seeing a bizarre issue that presents only on the iPhone 8 or iPhone 8 Plus, and I'm hoping someone here can suggest a reason for it.


What I am observing on the iPhone 8 and iPhone 8 Plus only, is that when I start my capture session, with the device in portrait or landscapeLeft orientation, then AVFoundation only reports 0 for the yawAngle of any faces it detects.


If I start the capture session with the the device in another orientation, or if I tilt the device into another orientation and then back to portrait or landscaleLeft, then the AVFoundation reports face yawAngle as usual.


In the app where I'm seeing this problem, I am also gathering AVFoundation sample buffers at the same time but this should not be a problem. In addition, I'm managing two CMMotionManagers, but I'm not using motion values to modify the capture session in anyway, so I don't think it's my logic that's producing this result. Also, none of my logic cares about UIDeviceOrientation as such, since it's observing much smaller angular changes.


Since it's the initial state of UIDeviceOrientation that produces this problem, and a change back and forth in UIDeviceOrientation that can fix it, it seems that the problem is produced by some mechanism that is watching UIDeviceOrientation itself, as I suppose AVFoundation must be doing, since it must know about the device's orientation in order to configure its face detector.


Alexis

Replies

To provide more detail on this, based on more experimentation:


It's only observed on the iPhone.8 and iPhone 8 Plus. The same code behaves differently on an iPhone 7.


When I start a capture session with a metadata data outputs, with the device in portrait mode, I get incorrect faceYaw data (zeros).

However, if I turn my head 90 degrees counterclockwise along the axis from my nose to the phone, and then turn my head left and right, it does show face yaw. In other words, at least as regards the faceYaw values, AVFoundation is behaving as if it thinks the device is LandscapeLeft instead of portrait. Probably not coincidentally, this is also the orientation reported by the data output object's connection's videoOrientation.


Another way to fix this is to briefly re-orient the device away from portrait, then back to protrait. This fixes it.


So it seems like the iOS 11 AVFoundation face detector code, only on an iPhone 8, behaves as if it has been configured with an incorrect orientation, on iPhone 8 but not other phones.


I'd love any suggestions sa to why.

I found a workaround. I'm pretty sure this is an Apple bug. I filed a a radar: 35064476


Alexis

can you share your workaround ? i faced with same issue.

thanks