VisionOS IBL (ImageBasedLighting) BW only or Coloured? File formats? Shadows?

Question

Created Dec ’23

Replies 3

Boosts 1

Views 1.6k

Participants 2

I have an immersive environment with a skybox which uses an png image inside a sphere. I added an IBL, but I am not sure what the best format / prep method is for the IBL image.

I have tried several different images for my IBL, and all are very different vibes from what I have in Blender.

My question is how can I create an IBL that's closest to Blender's Cycles rendering engine? However, it's a rather difficult to answer question, so I want to ask some smaller questions first.

Does IBL need to be BW or will colour work?
- From my tests: colour works just as well. But why does Apple only show use of BW ones? Should we use BW?
What is the best file format for IBL? Any pros/cons? Or should we just test out each format and check visually.
- From my tests: PNG, OpenEXR (.exr), Radiance HDR (.hdr) all work. But which format is recommended?
Will IBL on visionOS create shadows for us? In Blender an HDRI gives shadows.
- From my tests: No, IBL does not provide shadows on your loaded environment/meshes. Is "shadow baking" the only solution for the time being?
Looking at a scene in Blender which uses HDRI as global lighting, how can we best "prep" the IBL image that will give the closest light similar to Blender's Cycles rendering engine?
- From my tests: I tried (as shown below)
- A) make a render of just the Blender HDRI (without meshes) via 360-degree panoramic camera.
  - →　Usage as IBL makes everything too bright.
- B) make a render of the entire Blender scene via 360-degree panoramic camera.
  - →　Usage as IBL makes everything washed out and yellowish.
- C) Use the Sunlight.png from the sample project.
  - →　With this IBL the scene is too dark.
- D) Use the SystemIBL.exr from another sample project.
  - →　With this IBL the scene looks very flat and not realistic at all.

Here I show each IBL I described above 1~4 and sample screenshots from the simulator:

A) 1. env only hdr.jpg

B) 2. full room exr.jpg

C) 3. sunlight.jpg

D) 4. system ibl exr.jpg

The atmosphere I'm aiming for as per Blender's Cycles rendering engine:

blender cycles.jpeg

Can anyone help me with my questions 1 ~ 4 above. It would give me some insight in how to create immersive environments with realistic lighting & shadows. : )

Much appreciated!

— Luca

Answered by mesqueeb in 778512022

To answer my own post with my current understanding on things:

1- Does IBL need to be BW or will colour work?

It can be in colour. As an example you can use Reality Converter to test multiple different IBLs, some b/w some coloured:

ibl with color 1.jpg ibl with color 2.jpg

2- What is the best file format for IBL? Any pros/cons? Or should we just test out each format and check visually. From my tests: PNG, OpenEXR (.exr), Radiance HDR (.hdr) all work. But which format is recommended?

It "depends". Just test and see until you have the ambience you are looking for.

3- Will IBL on visionOS create shadows for us? In Blender an HDRI gives shadows.

No. There is currently no clear information what the best way is to approach adding shadows to a scene in RealityKit.

4 Looking at a scene in Blender which uses HDRI as global lighting, how can we best "prep" the IBL image that will give the closest light similar to Blender's Cycles rendering engine?

Just test and see until you have the ambience you are looking for.

Boost

Answer 1

Wittmason OP

Jan ’24

Hi Luca I can't offer any suggestions but I am curious on what you mean by "ShadowBaking"? On our Reality Kit composer objects (.usdz) they don't currently have any cast shadows.

Did you find a solution to this in the shadow baking you mentioned?

0

Answer 2

mesqueeb OP

Feb ’24

@Wittmason with "shadow baking" I meant "light baking". What I really mean is that in the Blender renderer (the screenshots at the bottom) you can see beautiful shadows. They are not generated when running on the  Vision Pro.

An IBL will "tint" our assets' colour, but it will not add shadows. I think the only solution to have shadows is to replace the textures with textures that have those shadows baked in. I guess they call this "light baking" in Unity/Unreal, but Blender doesn't have a good workflow for this.

My current understanding:

an asset texture as a JPG is sometimes a smaller image used in repeated pattern on an asset
other times an asset texture is just colour/texture data, not using any jpg

In consideration of the above, what I understand the limitations of light baking to be:

When baking light onto a JPG texture (so you can see a shadow as part of that texture), then you basically need one big texture jpg per asset and cannot use any "repeated pattern" any more.
When baking light onto a texture that doesn't use any JPG but instead colour/texture data, then a newly JPG will need to be generated for that.

The above 1 & 2 makes it so the usdz file size will grow tremendously because of the new and large sizes texture JPGs...

I believe that Unity/Unreal therefore don't "bake light onto a texture" but instead they create a "lightmap" which is another layer that can be added on top of an asset with data on which parts need to be lightened/darkened. I am guessing this is a more economical method to do so, but I have no idea how these lightmaps are supported by RealityKit /  Vision Pro.

0

Answer 3