How do we author a "reality file" like the ones on Apple's Gallery?

Hi @d0g

Hope this helps.

I’m not the most knowledgeable person on this subject but hopefully this is somewhat helpful.

For USDZ animation…

1. You can use various 3D applications for this. I’ve used Blender to create 3D models with simple, looping, animations with minimum movement in 3D. I use Reality Composer on the iPad to initiate larger movements of the simply animated 3D object in 3D space. 

I believe that this might also be able to be done using the new Reality Composer Pro app found in the new Beta Xcode 15.1 (Developer Tools) but it requires one to have an Apple silicon Mac to run It which I don't have. (It wasn't a requirement a week or so ago, but apparently now it is, which I'm not happy about.) I guess it’s time to upgrade…

2. From within Blender, or other DCC, export the animated file as .usdz preferably.

3. Open Reality Composer on the iPad, create a new file, and import (+ sign) your exported .usdz file. If you view it in AR now it won’t move yet.

4. Using Behaviors in Reality Composer, generate your AR experience with the imported .usdz file.

5. To move the object within the scene, and have it animate as defined in Blender, insert a Behavior with a “Custom” trigger and select which one you want to use to initiate the experience (Tap, scene start, etc.)

6. Add an “Action Sequence” card select “USDZ Animation” and choose your imported object.

7. Test run it. You should see the animation playing and the object will not move in x,y,z.

8. If you want it to also move while your .usdz animation plays, add another “Action Sequence” card and select the movement you want. (Eg. Move, Rotate, Scale To). Select the object, and set the position you want it to move to.

9. When you preview it, you will notice that the usdz animation plays first, and when finished it will move to the position defined. To have both actions happen at once, drag one action card onto the other to “combine” them.

For “Tapping on various buttons in this experience triggers various complex animations”.

1. In Reality Composer (iPad), you can select various items and have them trigger various behaviors such as mentioned above. It’s easy to have one item with a behavior on it to trigger movement of another object. Similar approach the notes above.

For “Exporting a Reality File” from Xcode, sorry, I’m no help here. You can do it from within the iPad application though…

1. With your file open in RC, select the ellipsis in the upper-right hand corner of the window and select “Export”. In the dialog box that opens, select "Reality File". The default format is "USDZ"
2. Export your file.

Hopefully that helps, cheers!

@CDNinCA That's super helpful, thanks!!

1. I've been trying for DAYS AND DAYS to ask Apple Support where to get Reality Composer Pro – and all they could tell me was I needed Vision Pro to use it. God I hope you're right and that it's bundled with Xcode 15.1 beta.

2. Regarding animations ... My modeler guy gives me a GLB file (containing GTLF) of a robot which has 3 animations inside, e.g., "WaveHand", "IdleLoop" etc. When I convert that to USDZ, and load in Reality Composer, I can play only 1 of the animations. How can I trigger different ones?

3. Regarding tapping... This same robot model has a few buttons which are part of the same model. In other workflows/apps I would make an invisible "hotspot" which is a 3D shape to capture taps. How can I make taps on different parts of this robot trigger different actions?

Thanks again!

. @d0g

Glad to be of some help.

I'd maybe get the most recent version of the Xcode beta, I think it’s now 15.2? If you have a Mac with Apple silicon in it you should be able to run RC Pro. Find it under “Xcode ->Developer Tools”. I wish I could still access it but I only have an Intel-based Mac.

My coding skills are really not where I would like them to be. Maybe there’s another way to do this in code than this idea, wouldn’t be surprised if there is, but here you go…

Regarding animations, USDZ animations are limited, usually only one thing at a time, or maybe two at most. It’s not a robust system. For example, you could have a propeller spinning on an airplane model while the plane slips side to side in a repeating animation loop. Nothing too complex. So I don’t know if the three animations will work, and there’s no way to control one, then the other independently. Once again, simple animations only.

You might try creating two usdz files, yes this sounds like a serious kludge but it might work? One file will have the “WaveHand” animation, another will have the “IdleLoop” or whatever else you might want. Export both files as .usdz.

In a new file in RC on the iPad, create a couple of scenes. The first one could have the usdz animation of the waving hand, another could have a different animation in it. If you create the first one, then duplicate the scene, and replace the model with the different animation, they will register in the same place when switching scenes. So each scene would have a different animation in it.

This might not be what you want but if you had a “button”, or some other item that triggers the other animation, you could have a behavior that changes scenes so the other animation would play. Use the “Custom” behavior, and add a “Change Scene” action card to it which points to the other scene. Hopefully that makes sense?

As for your last question, once again this seems awkward to me too but it might work. I sure hope that Apple continues to improve these applications….

Basically, in the 3D application, you split the model into two parts, say a button (only), and the rest of the model. So delete everything in the model except the button, and export it as .usdz. Then open the original model again, delete the button, and export the file. So you end up with two files. If the parts have the same orientation points when you export them you should be able to import them into RC and have them register with each other correctly, as they would if it was one model file. Make sense?

Then, in RC you can select the button without effecting the larger complete model, and assign behaviors to it independently.

Once again, I hope that makes sense. It seems that until these tools continue to improve one has to get “creative” in order to hopefully get things to work. It’s a fun challenge, but one that I hope goes away over time.

Cheers!

Reality Composer (not Pro) is not included with the newest release of Xcode, and there is no option to download it from developer resources for macOS. The most recent Xcode version that I am aware of that includes Reality Composer for macOS is 14.3.1, however it is incompatible with Sonoma. You can still access it through the "Open Developer Tool" menu. Once you have the app, you can either export the ".rcproject" as ".reality" directly, or incorporate it into a Xcode project, which will automatically convert it during compilation. I hope this is helpful. I still have some existing projects that used this format, so I am presently in the process of phasing out support for it and solely utilizing USDZ instead.

I've wondered the same thing. I'd hoped Reality Composer Pro would allow me to look at the file and see how it was created, but that's sadly not the case. I may be wrong, but I think The Hab was also created in Reality Composer, before Pro came on the scene.

I'd love to see more from the team about how it was created and assembled it.

In my short time with both, Reality Composer and Reality Composer Pro feel way more different than I assumed they would. I'm a little frustrated that "Pro" doesn't seem to offer many of the options of the base Reality Composer experience.