PHPickerViewController & photo metadata

Hi folks,

I work on a free open source/open science app called iNaturalist, we use CoreML / computer vision to identify photos of wildlife, and users can turn those photos into geotagged occurrence records that scientists can use to study species in space and time. This preface is my way of explaining that I care a lot about getting accurate metadata for the photos my users share with our app.

In iOS 14, I'm playing around with PhotosUI and PHPickerViewController and it seems that the only way to get photo metadata from the photo a user picks is by using the PHPhotoLibrary configuration. If I don't use the PHPhotoLibrary configuration when initializing the PHPickerViewController, my app will get UIImages that have no metadata in the underlying cgImage, and the assetIdentifier field is nil.

When using the PHPhotoLibrary configuration when initializing PHPickerViewController, I'm seeing some unexpected (to me) behavior. It would be helpful if someone else can confirm this, and confirm that this is behaving as intended.

  • if the user grants no access to the photo library, when the user selects a photo with PHPickerViewController, my app gets a UIImage but no PHAssetIdentifier, so no metadata at all. This seems totally reasonable - the user has chosen not to share full image metadata with me.

  • if the user has granted full permissions to the photo library, when a user selects a photo with PHPickerViewController my app gets both a UIImage and a PHAssetIdentifier which can be used to fetch a PHAsset via the PHPhotoLibrary. Using the PHAsset I can get metadata directly from the asset properties (this can be problematic, see note below) or by loading the EXIF via CoreImage properties with the method [asset requestContentEditingInputWithOptions:completionHandler]. This also seems totally reasonable, the user has chosen to share full image metadata with me.

  • if the user grants limited access to the photo library, things get interesting. When the user selects a photo with PHPickerViewController, my app gets a UIImage and a PHAssetIdentifier.

    • If the photo is part of the limited access set for my app granted by the user, the PHAssetIdentifier will be valid and can be used to fetch a PHAsset, which can then be used to extract metadata as described in the previous bullet point. For this photo, the behavior is the same as if the user has granted full access to the photo library. This seems reasonable - the user has chosen to share full image metadata for this photo with me.

    • If the photo is not part of the limited access set for my app granted by the user, the PHAssetIdentifier can be used to create but not fetch a PHAsset. I still see the fields on the PHAsset (location, datetime, etc) but I cannot fetch it from the PHPhotoLibrary - I get the error The operation couldn’t be completed. (Cocoa error -1.) So in this final case I get the selected UIImage, and a glimpse of the photo metadata as mediated (sometimes incorrectly, again see note below) by the PHAsset. This doesn't seem reasonable - the user has not chosen to share full image metadata for this photo with me, but some image metadata is leaked via the PHAsset properties.

    • If the photo is not part of the limited access set, the user will be prompted by iOS to change the limited access set or keep it the same. This creates the scenario where a user is asked twice to select a photo to share with my app, once to pick the photo and then again to grant permissions to it. This is a little cumbersome but I'm happy to jump through some extra hoops to preserve my users privacy.


Note that PHAssets always have a creation date, which might be an EXIF taken date. However, if the photo didn't have an EXIF taken date (it taken by another app or another device that doesn't save EXIF and then copied to the iPhone via iCloud or other photo sync mechanisms, for example), the creation date will be the date that it was imported into the Photo Library. So the PHAsset creation date is not 100% trustworthy for me if I'm trying to make records to be used by scientists to understand when an organism was seen. EXIF has problems but it's much more reliable.

Thanks for reading,
alex
Up vote post of alexshepard Down vote post of alexshepard
5.4k views
Posted by
alexshepard
Reply
Add a Comment

Accepted Reply

You can access asset source files using itemProvider.loadFileRepresentation and get most metadata without requesting Photo Library permission. Please refer to this thread for more information.

If the photo is not part of the limited access set for my app granted by the user, the PHAssetIdentifier can be used to create but not fetch a PHAsset.

Can you explain "can be used to create but not fetch a PHAsset" in detail?
Posted by
Frameworks Engineer
Up vote reply of Frameworks Engineer Down vote reply of Frameworks Engineer
Post marked as solved
Add a Comment

Replies

You can access asset source files using itemProvider.loadFileRepresentation and get most metadata without requesting Photo Library permission. Please refer to this thread for more information.

If the photo is not part of the limited access set for my app granted by the user, the PHAssetIdentifier can be used to create but not fetch a PHAsset.

Can you explain "can be used to create but not fetch a PHAsset" in detail?
Posted by
Frameworks Engineer
Up vote reply of Frameworks Engineer Down vote reply of Frameworks Engineer
Post marked as solved
Add a Comment
Hi @alexshepard,

first of all, cool app! I'm one of your users!

Apple has been suggesting to replace custom made pickers by PHPickerViewController. In your situation, I think it makes sense.
Your user can use off the shelf filters like animals or flowers to help finding images. Then you can use file or data representation as previously mentioned to access exif data.
Although it is a noble idea, limited access mode can force an awkward flow for the user. If you can avoid it, just do it!
Because date taken is super important for your app, if you don't have exif data, you should not trust on the date of the file at all. If you don't have exif, accessing creation date either through PhotoKit or PHPickerViewController can have completely wrong dates. If your user has a camera, copies files to a computer and then sends them to iCloud, there is a high likeliness of having a file with an incorrect date. In this situation, It makes sense to request it to be manually added.
It is also important to note that accessing raw data, may force your user to pull it from the cloud (iCloud). If your user selects 4 or 5 images, that can be something like 10 MB to be downloaded. Your UI should be prepared for it accordingly. The good thing is that you then can use those bytes to upload the images to your server.

cheers
Posted by
_mc
Up vote reply of _mc Down vote reply of _mc
Add a Comment
Hi Frameworks Engineer,

Thanks so much! That's exactly what I wanted.

Can you explain "can be used to create but not fetch a PHAsset" in detail?.

Sure, I can explain but it may have been that since I misunderstood how to use the Picker, I ended up in a workflow that doesn't make any sense. I also should not have said "but not fetch", I should have said "the UIImage for the PHAsset cannot be fetched from PHImageManager." I can see now that this shouldn't be necessary anyways - the image is available from the itemProvider.

Here's some sample code:
Code Block swift
class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()
        PHPhotoLibrary.requestAuthorization { (status) in
            print("\(status)")
        }
    }
    
    @IBAction func library(sender: UIButton) {
        let config = PHPickerConfiguration(photoLibrary: PHPhotoLibrary.shared())
        let picker = PHPickerViewController(configuration: config)
        picker.delegate = self
        self.present(picker, animated: true, completion: nil)
    }
}
extension ViewController: PHPickerViewControllerDelegate {
    func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {
        dismiss(animated: true, completion: nil)
        if let assetId = results.first?.assetIdentifier,
           let asset = PHAsset.fetchAssets(withLocalIdentifiers: [assetId], options: nil).firstObject
        {
            print("asset is \(asset)")
            print("asset location is \(asset.location)")
            PHImageManager.default().requestImage(for: asset, targetSize: CGSize(width: 100, height: 100), contentMode: .aspectFit, options: nil, resultHandler: { (image, info) in
                print("requested image is \(image)")
            })
        }
    }
}


Assuming you have set a value in the Info.plist for Privacy - Photo Library Usage Description, then when you launch the app in the simulator, iOS will present the photo permission alert. Let's say that you choose "Select Photos..." and select only one photo (say one of the waterfall photos).

Then, you trigger the IBAction and open the PHPickerViewController. If you choose the waterfall image that you previously chose to share with the app, you will see printed in the logs the asset, the asset location, and the image requested from PHImageManager.

However, if you choose one of the photos that you did not choose to share with the app, you will see printed in the logs the asset and the asset location, but the image requested from PHImageManager will be nil and there will be a bunch of errors about Invalid asset uuid for client.

Again, I think I only ran into this because I was using the PHPickerViewController incorrectly, so I don't think it's a real problem.

Thanks again for your help!

Best,
alex

Posted by
alexshepard
Up vote reply of alexshepard Down vote reply of alexshepard
Add a Comment
Hi @_mc,

Thanks for the kind words!

After looking at the link provided by the frameworks engineer and also your comments, I've done a first implementation of PHPickerViewController for our users. So far it seems to be working great, but I'll be sure to keep an eye on iCloud fetches and UI / performance implications.

Thanks,
alex
Posted by
alexshepard
Up vote reply of alexshepard Down vote reply of alexshepard
Add a Comment