iOS 11 bug with cinematic video stabilization

I notice a horrible bug in iOS 11. If I set a AVCaptureVideoDataOutput and set the video connection stabilization mode to .cinematic, then the sample buffers that come out have green/red bars on the top. I am wondering if any known workaround is available. It's pretty easy to reproduce actually.

Replies

Dear Apple Engineering team, I filed a bug #34591156 for the above issue. Please do look into it and let me know if there are workarounds available.

HI Devgeek.


Thank you for filing the bug report. We have root-caused the problem and are working on a fix. Affected platforms are iPhone 7, 7 Plus, 8, and 8 Plus. As a temporary workaround, you can switch to the less aggressive "Standard" stabilization mode.

Hello bford,

Any idea if this would be fixed in iOS 11.0.1 or so? Because we are unnecessarily getting customer emails and app reviews complaining they can't use Cinematic stabilization anymore.

@bford Similar issue is also seen on iPhone 8 plus with 4K preset with 'Standard' video stabilization setting. Seems like video stabilization is thoroughly broken on iOS 11 with VDO.

On iPhone 8, standard video stabilization suffers from the same problem at the 4K30 resolution, but should be unaffected at lower resolutions. It's the same bug.

It's still not fixed in iOS 11.1 beta 2. Any possibility of it getting fixed in iOS 11.1 final release?

This seems to be fixed in iOS 11.2 beta 2. I've done some testing now with .cinematic and I don't seem to have the bars at the top anymore. This is on an iPhone 7. I'm not sure if it's been fixed for .standard with the 4K preset.

@bford I have been testing iOS 11.2 betas and can see the green bar is gone. BUT the issue with iPhone 8/X 4K resolution is still there. The video frames are distorted and tiled with standard video stabilization. You said the bug is same, why is this issue not fixed?

To track 4K@30 fps issue, I have filed a bug #35488838. If there are workarounds, let me know.

@bford I filed a bug#35488838 for 4K@30 fps video frames corruption with standard video stabilization along with sample code. To my surprise I am asked to provide a sample video and sysdiagnose log. I thought the engineering is already aware of the issue as you mentioned it few weeks before.

4K@30 fps hasn't been fixed unfortunately. And it's highly unlikely to be fixed in iOS 11.2 release as it's still not fixed in beta 3. I am not sure what is a workaround, may be skipping every alternate frame perhaps with 60 fps.

Ok so the bug is marked as dup of another one which is still open. I intend to implement a workaround therefore. I need to know how can one emulate 4K@30 fps preset using 4K@60 fps. Since 4K@60 fps works with video stabilization, is simply ignoring every alternate video frame from 60 fps stream an acceptable workaround? In otherwords, will it match 4K@30 fps in terms of natural exposure settings, stabilization quality, and smoothness?

@bford Bug #35488838 continues to remain open and all the betas including iOS 11.2.5 beta 6 also suffers with it. There are no known workarounds with 4K + dual lens with video stabilization set to 'Standard'.

I think engineering wanted to verify that the problem we identified was indeed visually the same issue that you described. The problem has been root caused and will be addressed in a future release.