Post not yet marked as solved
The methods described in https://developer.apple.com/forums/thread/715452?answerId=729571022#729571022 to obtain 48 MP image captures no longer seem to work on iOS 17.4 under certain circumstances.
Previously, the following steps were sufficient to get 48 MP capture from AVFoundation:
Configuration
Set the active AVCaptureDevice.Format to a format where supportedMaxPhotoDimensions contains the (8064, 6048) size
Set AVCapturePhotoOutput.maxPhotoDimensions to (8064, 6048)
Set AVCapturePhotoOutput.maxPhotoQualityPrioritization to .quality
Taking a photo
Set AVCapturePhotoSettings.maxPhotoDimensions to (8064, 6048)
Set AVCapturePhotoSettings.photoQualityPrioritization to .quality
As of iOS 17.4, the exact same code that worked through 17.3 no longer works if the session was configured manually (resulting in the .inputPriority session preset) rather than using a session preset (like .high). When configuring the session manually, all the intervening steps work (an active format can be found with the appropriate dimensions, the photo output settings can be set to 8064x6048 successfully, etc.), but the resulting photo is 4032x3024. Again, these same steps worked flawlessly prior to iOS 17.4.
Am I missing something? Did iOS 17.4 change the requirements for 48 MP capture, or is this a bug?
Post not yet marked as solved
We are using AVAssetWriter to write videos using both HEVC and H.264 encoding. Occasionally, we get reports of choppy footage in which frames appear out of order when played back on a Mac (QuickTime) or iOS device (stock Photos app). This occurs extremely unpredictably, often not starting until 20+ minutes of filming, but occasionally happening as soon as filming starts.
Interestingly, users have reported the issue goes away while editing or viewing on a different platform (e.g. Linux) or in the built-in Google Drive player, but comes back as soon as the video is exported or downloaded again. When this occurs in an HEVC file, converting to H.264 seems to resolve it. I haven't found a similar fix for H.264 files.
I suspect an AVAssetWriter encoding issue but haven't been able to uncover the source.
Running a stream analyzer on HEVC files with this issue reveals the following error:
Short-term reference picture with POC = [some number] seems to have been removed or not correctly decoded.
However, running a stream analyzer on H.264 files with the same playback issue seems to show nothing wrong.
At a high level, our video pipeline looks something like this:
Grab a sample buffer in captureOutput(_ captureOutput: AVCaptureOutput!, didOutputVideoSampleBuffer sampleBuffer: CMSampleBuffer!)
Perform some Metal rendering on that buffer
Pass the resulting CVPixelBuffer to the AVAssetWriterInputPixelBufferAdaptor associated with our AVAssetWriter
Example files can be found here: https://drive.google.com/drive/folders/1OjDZ3XaC-ubD5hyDiNvMQGl2NVqZbWnR?usp=sharing
This includes a video file suffering this issue, the same file fixed after converting to mp4, and a screen recording of the distorted playback in QuickTime.
Can anyone help point me in the right direction to solving this issue? I can provide more details as necessary.
Post not yet marked as solved
The latest iPhone 15 Pro models support additional focal lengths on the main 24mm (1x) lens: 28mm ("1.2x") and 35mm ("1.5x"). These are supposed to use data from the full sensor to achieve optical quality images (i.e. no upscaling), so I would expect these new focal lengths to appear in the secondaryNativeResolutionZoomFactors array, just like the 2x option does. However, the activeFormat.secondaryNativeResolutionZoomFactors property still only reports [2.0] when using the main 1x lens. Is this an oversight, or is there something special (other than setting the zoom factor) we need to do to access the high-quality 28mm and 35mm modes? I'm wary of simply setting 1.2 or 1.5 as the zoom factor, as that isn't truly the ratio between the base 24mm and the virtual focal lengths.
When capturing RAW (not ProRAW) photos using AVCapturePhotoOutput, the resulting images are subject to a strange overexposed effect when viewed in Apple software. I have been able to recreate this in multiple iOS apps which allow RAW capture.
Some users report previously normal images transforming over the span of a few minutes. I have actually watched this happen in real-time: if you observe the camera roll after taking a few RAW photos, the highlights in some will randomly **** (edit: this just says b l o w, nice job profanity filter) out of proportion after whatever is causing this issue kicks in.
The issue can also be triggered by zooming in to one of these images from the stock Photos app. Once the overexposure happens on a given image, there doesn't appear to be a way to get it to display normally again. However, if you AirDrop an image to a different device and then back, you can see it display normally at first and then break once more.
Interestingly, the photo displays completely fine when viewed in Affinity photo or random photo viewers on Ubuntu.
Sometimes the issue is not that bad, but it is often egregious, resulting in completely white areas of a previously balanced photo (see https://discussions.apple.com/thread/254424489).
This definitely seems like a bug, but is there any way to prevent it? Could there be an issue with color profiles?
This is not the same issue in which users think RAW photos are broken because they are viewing the associated JPG – this happens even with photos that have no embedded JPG or HEIC preview.
Very similar (supposedly fixed) issue on MacOS:
https://www.macworld.com/article/1444389/overexposed-raw-image-export-macos-monterey-photos-fixed.html
Numerous similar complaints:
https://discussions.apple.com/thread/254424489
https://discussions.apple.com/thread/253179308
https://discussions.apple.com/thread/253773180
https://discussions.apple.com/thread/253954972
https://discussions.apple.com/thread/252354516
tenuki.