live updates of spatial image metadata changes

Question

Created Jun ’24

Replies 2

Boosts 0

Participants 2

Session 10166 goes into wonderful detail about the construction of spatial photos, and the various parameters that define the relationship between left and right images. The session provides everything I need to know to combine a left and right frame, and create a spatial image output.

But I'd like to do a live preview on the Vision Pro. Change the baseline and see what it looks like. See the horizontal disparity/convergence adjustment and move the image back and forth.

Cropping, and vertical alignment, would be easy to implement live. Horizontal disparity, and baseline length? I'm baffled.

How would I create a Shader Graph to let me make these adjustments using sliders or similar affordances, and pipe the results to a Camera Index Switch? I already have a working stereography app, but the stereo parameters are not interactive at all.

I could regenerate the spatial image after each change and refresh the display, but that is awfully clunky. What's a better way?

Boost

Answer 1

halm OP

Jun ’24

Thinking about this some more...I can control horizontal disparity with a slider, specifically using it to control the horizontal centers of the right/left images. The only remaining piece is the baseline length, but that's fixed on Vision Pro so not really worth tinkering with in the spatial image.

0

Answer 2

Vision Pro Engineer OP

Apple

Jun ’24

If you're using custom rendering in Shader Graph, controlling horizontal disparity by adjusting the UVs of the left and right images sounds like a great idea! This will simulate what happens in the system spatial renderer as you adjust the metadata field.

However, please note that in the system spatial media renderer, the image plane is behind a window. And the perceptual effect of a disparity shift, which is relative to the image size, can vary depending on scale, distance, and position of the user. So while you can create a great experience rendering a spatial photo to a plane, if you want to preview exactly how your content would look in the Photos app, I'd recommend using PreviewController API to view the spatial photo with the same treatment.

0