@Vision Pro Engineer thank you for the sample and suggestion to use an ECS system to do so.
I was hoping for an action similar to the fromToByAction:
https://developer.apple.com/documentation/realitykit/fromtobyaction
that we can apply to the intensity, just like we can with other properties like position etc.
Do you know if this is possible?
Post
Replies
Boosts
Views
Activity
@isomoon @rakesha @steve23
Take a look at my article here:
https://fatbobman.com/en/posts/merging-swift-string-catalogs-in-a-multi-package-monorepo/
I have the same issue. Does this mean that any visionOS apps cannot use Xcode Cloud?
@Lucky7 Did you figure it out? I was wondering the same thing.
To answer my own post with my current understanding on things:
1- Does IBL need to be BW or will colour work?
It can be in colour. As an example you can use Reality Converter to test multiple different IBLs, some b/w some coloured:
2- What is the best file format for IBL? Any pros/cons? Or should we just test out each format and check visually. From my tests: PNG, OpenEXR (.exr), Radiance HDR (.hdr) all work. But which format is recommended?
It "depends". Just test and see until you have the ambience you are looking for.
3- Will IBL on visionOS create shadows for us? In Blender an HDRI gives shadows.
No. There is currently no clear information what the best way is to approach adding shadows to a scene in RealityKit.
4 Looking at a scene in Blender which uses HDRI as global lighting, how can we best "prep" the IBL image that will give the closest light similar to Blender's Cycles rendering engine?
Just test and see until you have the ambience you are looking for.
@Wittmason with "shadow baking" I meant "light baking". What I really mean is that in the Blender renderer (the screenshots at the bottom) you can see beautiful shadows. They are not generated when running on the Vision Pro.
An IBL will "tint" our assets' colour, but it will not add shadows. I think the only solution to have shadows is to replace the textures with textures that have those shadows baked in. I guess they call this "light baking" in Unity/Unreal, but Blender doesn't have a good workflow for this.
My current understanding:
an asset texture as a JPG is sometimes a smaller image used in repeated pattern on an asset
other times an asset texture is just colour/texture data, not using any jpg
In consideration of the above, what I understand the limitations of light baking to be:
When baking light onto a JPG texture (so you can see a shadow as part of that texture), then you basically need one big texture jpg per asset and cannot use any "repeated pattern" any more.
When baking light onto a texture that doesn't use any JPG but instead colour/texture data, then a newly JPG will need to be generated for that.
The above 1 & 2 makes it so the usdz file size will grow tremendously because of the new and large sizes texture JPGs...
I believe that Unity/Unreal therefore don't "bake light onto a texture" but instead they create a "lightmap" which is another layer that can be added on top of an asset with data on which parts need to be lightened/darkened. I am guessing this is a more economical method to do so, but I have no idea how these lightmaps are supported by RealityKit / Vision Pro.
GroundingShadowComponent(castsShadow: true) needs to be applied to the first layer you have that actually has the mesh. In Apple language sometimes called the ModelEntity.
Probably bigDonut has an extra /Root layer, so you need to dig into the children, find the ModelEntity and apply the shadow there.
@mzoob Can you please share the image you used for "shiwai_a" ? I'm having difficulties creating the correct .exr/.hdr/.png System IBL Texture to achieve the same lighting as I see it in Blender in my room & garden. :sweat_smile:
@gchiste
Do you have other examples of IBL that I can reference to try and understand how to better add lighting to my scenes for the full immersion mode?
I followed the video and checked the example code.
I also found this example code on Stack Overflow: https://stackoverflow.com/questions/76755793/spotlightcomponent-is-unavailable-in-visionos/76761509#76761509
I have a scene with a bunch of lights in Blender, but none of them auto translate to IBL and I'm not sure how/where to start learning about how to convert my Blender lights to IBL for visionOS.
I can't find any other examples to better understand how to create an IBL for a lantern here or the sun there as I can see it in my Blender scene. Any official direction from Apple would be greatly appreciated!
How about these:
https://developer.apple.com/documentation/realitykit/modifying-realitykit-rendering-using-custom-materials
and
https://developer.apple.com/metal/Metal-RealityKit-APIs.pdf
I hope it helps
@kevyk yes, but the issue was that applyLinearImpulse doesn't exists on the parent entity. I was able to get around it by doing this:
let parentEntityHasPhyics = parentEntity as? HasPhysicsBody {
parentEntityHasPhyics.applyLinearImpulse(...) // now we can use applyLinearImpulse without getting a compile-time error
}
@gchiste What if we have a transform with 3 ModelEntities inside in reality composer pro. Something like this:
and since the three model entities represent a single object, can I execute applyImpulse to the "transform" layer named Body or do I have to do it to each individual model entity? (those three nested ones)
Because I believe my transform layer called "Body" is technically an Entity, and not a ModelEntity, right?
Did you check out the Happy Beam sample project? You can pretty much reference that to do what you wanna do.
@Gong thanks for the message. But I'm still not sure when we should apply grounding shadow, and to which entities.
I believe you can achieve this by making a separate usda file of your entity: right click the entity, and choose "New Scene From Selection".
Make sure you have 1 root transform. This root's origin will be at the center of the scene. Now you drag your model entity (a child of this root) to the spot you want.
Then in the original file, your entity has become a reference to that new scene you made. But rotating it now should keep the origin as per your new scene.