How to read back from Spatial Image encoded with HEIC information about which image at which index is left or right?

Question

Created Jun ’24

Replies 1

Boosts 0

Participants 2

In the example https://developer.apple.com/documentation/imageio/writing-spatial-photos, we see that for each image encoded with the photo we include the following information:

kCGImagePropertyGroups: [
    kCGImagePropertyGroupIndex: 0,
    kCGImagePropertyGroupType: kCGImagePropertyGroupTypeStereoPair,
    (isLeft ? kCGImagePropertyGroupImageIsLeftImage : kCGImagePropertyGroupImageIsRightImage): true,
    kCGImagePropertyGroupImageDisparityAdjustment: encodedDisparityAdjustment
],

Which will identify which image is left, and which is right, also information about group type = stereo pair.

Now, how do you read those back?

I tried to implement a reading simply with CGImageSourceCopyPropertiesAtIndex, and that did not work, getting back "No property groups found."

func tryToReadThose() {
    guard
        let imageData = try? Data(contentsOf: outputImageURL),
        let source = CGImageSourceCreateWithData(imageData as NSData, nil)
    else {
        print("cannot read")
        return
    }
    
    for i in 0..<CGImageSourceGetCount(source) {
        guard let imageProperties = CGImageSourceCopyPropertiesAtIndex(source, i, nil) as? [String: Any] else {
            print("cannot read options")
            continue
        }
        
        
        if let propertyGroups = imageProperties[String(kCGImagePropertyGroups)] as? [Any] {
            // Process the property groups as needed
            print(propertyGroups)
        } else {
            print("No property groups found.")
        }
        
        //print(imageProperties)
    }
}

I assume maybe CGImageSourceCopyPropertiesAtIndex expects something as a 3rd parameter. But in the https://developer.apple.com/documentation/imageio/cgimagesource "Specifying the Read Options" I don't see anything related to that.

Answered by Vision Pro Engineer in 791139022

You're on the right track. The next steps are:

Find the group that contains the stereo pair.
Extract the left and right index from that group.
Use the indexes to create the left and right images.

let imageSource = CGImageSourceCreateWithURL(url as CFURL, nil),
let cfProps = CGImageSourceCopyProperties(imageSource, nil)  {
let props = cfProps as Dictionary
                    
if let groups = props[kCGImagePropertyGroups] as? [[CFString:Any]] {
    let stereoGroup = groups.first(where: {
        let groupType = $0[kCGImagePropertyGroupType] as! CFString
        return groupType == kCGImagePropertyGroupTypeStereoPair
    })

    if let stereoGroup,
        let leftIdx = stereoGroup[kCGImagePropertyGroupImageIndexLeft] as? Int,
        let rightIdx = stereoGroup[ kCGImagePropertyGroupImageIndexRight] as? Int,
        let leftImage = CGImageSourceCreateImageAtIndex(imageSource, leftIdx, nil),
        let rightImage = CGImageSourceCreateImageAtIndex(imageSource, rightIdx, nil) {

        // Do things with the left and right images!
    }
}

Boost

Answer 1

Vision Pro Engineer OP

Apple

Jun ’24

Recommended

You're on the right track. The next steps are:

Find the group that contains the stereo pair.
Extract the left and right index from that group.
Use the indexes to create the left and right images.

let imageSource = CGImageSourceCreateWithURL(url as CFURL, nil),
let cfProps = CGImageSourceCopyProperties(imageSource, nil)  {
let props = cfProps as Dictionary
                    
if let groups = props[kCGImagePropertyGroups] as? [[CFString:Any]] {
    let stereoGroup = groups.first(where: {
        let groupType = $0[kCGImagePropertyGroupType] as! CFString
        return groupType == kCGImagePropertyGroupTypeStereoPair
    })

    if let stereoGroup,
        let leftIdx = stereoGroup[kCGImagePropertyGroupImageIndexLeft] as? Int,
        let rightIdx = stereoGroup[ kCGImagePropertyGroupImageIndexRight] as? Int,
        let leftImage = CGImageSourceCreateImageAtIndex(imageSource, leftIdx, nil),
        let rightImage = CGImageSourceCreateImageAtIndex(imageSource, rightIdx, nil) {

        // Do things with the left and right images!
    }
}

0