I'd like to implement the functionality to pause/play video playback when the user taps on the immersive video playback view. How can I handle the tap event on Vision OS?

I'd like to map a SwiftUI view (in my case: a map) onto a 3D curved plane in immersive view, so user can literally immersive themselves into the map. The user should also be able to interact with the map, by panning it around and selecting markers. Is this possible?

In visionOS, I want show a 3D Content, I can use RealityView or Mode3D, But the effect they achieve is similar. What is the difference between them and which one to use for users?

Hi, I'm working on a simple visionOS app and I'm testing on device. For one part of the app, I load an object in and place it on the user's hand. If I use a primitive shape, like a sphere or cylinder, this works fine. However, now I'm trying to load a an object from my RealityKitContent package. But everytime I try this, I get a an error message, resourceNotFound("Stone"), where "Stone" is one of my usda scenes. This is what the guts of my function looks like that should return a ModelEntity: do { let entity = try await ModelEntity(named: "Stone", in: realityKitContentBundle) entity.generateCollisionShapes(recursive: true) return entity } catch { print("Error \(error)") } I can see the "Stone" in my Xcode sidebar as part of the RealityKitContent package and inside that scene, there is a simple sphere, but alas I always get this in the Xcode console, "Error resourceNotFound("Stone")" I'm probably doing something pretty silly, hopefully it's obvious to someone else. Thanks for the help. Ian

In Reality Composter Pro has a triplanar projection node based on the provision of images. Is there a way to make a triplanar projection to input the dynamic material?

I have a main app window that presents an Immersive style in Mixed Reality. I am trying to determine the anchor/position of this glass window in the 3D space and place a Sphere entity right next to it. The goal is to ensure that if the user moves the window, the Sphere entity remains attached to it. Does anyone have insights on how to achieve this? The below code snippet provides the position of the device, and I have positioned it 0.5 meters away from the z-axis. However, my objective is to obtain the position of the glass window and anchor the sphere to it. Any guidance on achieving this would be appreciated. import RealityKit import RealityKitContent import ARKit struct ImmersiveView: View { let visionProPose = VisionProPose() var body: some View { RealityView { content in Task { await visionProPose.runArSession() } // Add the initial RealityKit content if let scene = try? await Entity(named: "Immersive", in: realityKitContentBundle) { content.add(scene) } } update: { content in if let scene = content.entities.first { if let sphere = scene.findEntity(named: "Sphere") as? ModelEntity { Task { let transfrom = await visionProPose.getTransform() sphere.position = [Float((transfrom?.columns.3.x)!), Float((transfrom?.columns.3.y)!), Float((transfrom?.columns.3.z)!) - 1 ] } } } } } } @Observable class VisionProPose { let session = ARKitSession() let worldTracking = WorldTrackingProvider() func runArSession() async { Task { try? await session.run([worldTracking]) } } func getTransform() async -> simd_float4x4? { guard let deviceAnchor = worldTracking.queryDeviceAnchor(atTimestamp: 1) else { return nil } let transform = deviceAnchor.originFromAnchorTransform return transform } }

Let's say I've created a scene with 3 models inside side by side. Now upon user interaction, I'd like to change these models to another model (that is also in the same reality composer pro project). Is that possible? How can one do that? One way I can think of is to just load all the individual models in RealityView and then just toggle the opacity to show/hide the models. But this doesn't seem like the right way for performance/memory reasons. How do you swap in and out usdz models?

Hey friends, I'm using a drag gesture to rotate a parent object that contains several child colliders. When I drag slowly, sometimes the child colliders don't rotate along with the parent. Any help would be appreciated, thanks! .gesture( DragGesture() .targetedToAnyEntity() .onChanged { value in let startLocation = value.convert(value.startLocation3D, from: .local, to: .scene) let currentLocation = value.convert(value.location3D, from: .local, to: .scene) let delta = currentLocation - startLocation let spinX = Double(delta.y) let spinY = Double(delta.x) let pitch = Transform(pitch: Float(spinX * -1)).matrix let roll = Transform(roll: Float(spinY * -1)).matrix value.entity.transform.matrix = roll * pitch })

Hello! I’m trying to make a material in RealityKit that has a basic gradient. I am making an iPadOS app. A few thoughts: I cannot use Reality Composer Pro to do this because the Shader Graph tool only works for visionOS. I cannot use a Metal file to create a shader because I am using a .swiftpm (app playgrounds) file targeted for Swift Playgrounds. Metal files don’t seem to work on Swift Playgrounds (it’s a Swift playground, after all). I would prefer to not use image textures for a simple thing like this. That would take up storage. I wish it was as easy as applying a .basecolor with a UIColor, but UIColor does not support gradients. What are my options? I know my requirements are likely not typical, but I really need to try to not break those. I looked into CustomMaterial from RealityKit but once again, those take Metal shaders. Amazing tool, but I sadly cannot use them because I’m using a Swift Playground that doesn’t seem to with Metal files, at least it seems. I’ve briefly done research on MetalKit? Could that help me out? Let’s say I have a simple box in RealityKit. How would I apply a simple gradient to it given my constraints? I really appreciate the help. P.S. This is a SwiftUI project for reference. EDIT: Could I create a texture without images, perhaps by making a view into a texture and applying it? How would I do this? What are the pros and cons of this? Another thought was could I just use MetalKit to create the gradient and apply it using CustomMaterial? I will say that I'm kind of at a last resort — I am trying to create fairly straightforward materials. The two most important ones would be a gradient type of material with two colors and a transparent water material with a bit of refraction and perhaps maybe a little bit of reflection. But, these are not meant to be photorealistic at all, not at all. They're meant to be fairly "2D"/simple which makes me wonder if I could just load in a texture. The only issue is that I don't know how I would do the water.

Dear Apple Developer Forum Community, I hope this message finds you well. I am writing to seek assistance regarding an error I encountered while attempting to create a "Tic Tac Toe" application using Xcode. Upon launching Xcode and starting a new project, I followed the standard procedure for creating a simple iOS application. However, during the process, I encountered I am trying to make an app but the code showing an error when any player won the match. I have attempted to troubleshoot the issue by see two images, but unfortunately, I have been unsuccessful in resolving it. I am reaching out to the community in the hope that someone might have encountered a similar issue or have expertise in troubleshooting Xcode errors. Any guidance, suggestions, or solutions would be greatly appreciated. Thank you very much for your time and assistance. Sincerely, Zipzy games

Hi, I tried to change the default size for a volumetric window but It looks like this window has a maximum width value. Is it true? WindowGroup(id: "id") { ItemToShow() }.windowStyle(.volumetric) .defaultSize(width: 100, height: 0.8, depth: 0.3, in: .meters) Here I set the width to 100 meters but It still looks like about 2 meters

Dear Apple Developer Forum Community, I hope this message finds you well. I am writing to seek assistance regarding an error I encountered while attempting to create a "Hello World" application using Xcode. Upon launching Xcode and starting a new project, I followed the standard procedure for creating a simple iOS application. However, during the process, I encountered an unexpected error that halted my progress. The error message I received was [insert error message here]. I have attempted to troubleshoot the issue by see two images, but unfortunately, I have been unsuccessful in resolving it. I am reaching out to the community in the hope that someone might have encountered a similar issue or have expertise in troubleshooting Xcode errors. Any guidance, suggestions, or solutions would be greatly appreciated. Thank you very much for your time and assistance. Sincerely, Zipzy games y Games

Hi all, I'm trying to retrieve the name of an entity from the gesture that hits it, but it's not giving me the value I set when I created the entity. I create the entity like: class DrumPad: ObservableObject { static func create(for audioFileName: String) -> Entity? { do { let padEntity = try Entity.load(named: "Geometry/pad-without-handle", in: tableDrummerContentBundle) padEntity.name = "\(audioFileName)_pad" return padEntity } catch { print("Could not load pad \(audioFileName)") print(error.localizedDescription) return nil } } } Then I get it from the gesture: var body: some View { RealityView { content in for sampleName in audioSamples { guard let pad = DrumPad.create(for: sampleName) else { continue } content.add(pad) } } .gesture(SpatialTapGesture() .targetedToAnyEntity() .onEnded { value in print(value.entity.name) }) } } In the gesture handler, print(value.entity.name) gives me the name of the root transform of the entity, PadTransform, not the string I set "\(audioFileName)_pad" during instantiation. If I call print(padEntity.name) during instantiation, I get rock-kick-2_pad and the like. Any help would be much appreciated.

Hello everyone, I want to develop an app for vision pro that aims to help people with vertigo and dizziness problems. The problem is that I can not afford vision pro. If I use standart vr set with an iPhone inside would it cause issues on real vision pro?

How to binding MTLTexture to Color input of the material? I need use something similar to VideoMaterial. So I need make a CustomMaterial. But RealityKit CustomMaterial is not available in VisionOS, and replaced by ShaderGraphMaterial So how to binding Metal resource such as MTLTexture to ShadeGraphMaterial directly.

Hello! I'm having a very odd problem. I was trying to open a USD file in Xcode so I could then open it in Reality Composer Pro. I've been able to do that without a problem for a number of weeks. However, I can't do that now. Every time I try to open a USD, Xcode briefly opens and then crashes. Then, every time I try to open Reality Composer Pro from the Developer Tools menu in Xcode, the app bounces up and down, opens for one second (little dot on the dock) and then just doesn't open. I have no idea what I did. I've been using Xcode 15.2 and all of the sudden it just doesn't work anymore. The only thing I could think of is that I used an online converter from GLB to USD and then tried opening up that USD, but the website was working for me before. Plus, when I try to open up other files like USDA, it still doesn't work. So, I don't think it's one type of file. I tried updating to macOS Sonoma 14.3.1 but that didn't fix it. Xcode is downloaded from the Mac App Store. I am not using any beta software. I tried doing the usual restart, clean build folder etc. but nothing works. I am really confused... all of the sudden it just stopped working. Any fixes? I am on a very tight deadline, and this app is crucial to my work. Thanks! :)

Hi, What are the limitations and capabilities of visionOS? I cannot find answers to the questions I have. Let's say you have some USDZ files stored in a cloud service, there are so many of them that the app would be huge if you put them in assets. You want to fetch the one you are interested in and show it while an app is running. Is it possible to load USDZ files at runtime from the network? Is there a limit to how many objects can be visible at once? Let's say I am in an open space, with no walls. I want to place 100 3D objects somewhere in space. Is it possible? What if I placed 500, 1000? Is there a way to save the anchor point of the object? I want to open the app again and have an object in the same place I left it. I would like to arrange my space and have objects always in the same spots. How does the OS behave if objects are in different rooms? Is it possible to walk around, visit different rooms, and have objects anchored there? Would it behave like real objects? Is it possible to color a plane? Let's say there is a wall and it's black. I want this wall to be orange. Is it possible?

Hi, I'm trying to display an STL model file in visionOS. I import the STL file using SceneKit's ModelIO extension, add it to an empty scene USDA and then export the finished scene into a temporary USDZ file. From there I load the USDZ file as an Entity and add it onto the content. However, the model in the resulting USDZ file has no lighting and appears as an unlit solid. Please see the screenshot below: Top one is created from directly importing a USDA scene with the model already added using Reality Composer through in an Entity and works as expected. Middle one is created from importing the STL model as an MDLAsset using ModelIO, adding onto the empty scene, exporting as USDZ. Then importing USDZ into an Entity. This is what I want to be able to do and is broken. Bottom one is just for me to debug the USDZ import/export. It was added to the empty scene using Reality Composer and works as expected, therefore the USDZ export/import is not broken as far as I can tell. Full code: import SwiftUI import ARKit import SceneKit.ModelIO import RealityKit import RealityKitContent struct ContentView: View { @State private var enlarge = false @State private var showImmersiveSpace = false @State private var immersiveSpaceIsShown = false @Environment(\.openImmersiveSpace) var openImmersiveSpace @Environment(\.dismissImmersiveSpace) var dismissImmersiveSpace var modelUrl: URL? = { if let url = Bundle.main.url(forResource: "Trent 900 STL", withExtension: "stl") { let asset = MDLAsset(url: url) asset.loadTextures() let object = asset.object(at: 0) as! MDLMesh let emptyScene = SCNScene(named: "EmptyScene.usda")! let scene = SCNScene(mdlAsset: asset) // Position node in scene and scale let node = SCNNode(mdlObject: object) node.position = SCNVector3(0.0, 0.1, 0.0) node.scale = SCNVector3(0.02, 0.02, 0.02) // Copy materials from the test model in the empty scene to our new object (doesn't really change anything) node.geometry?.materials = emptyScene.rootNode.childNodes[0].childNodes[0].childNodes[0].childNodes[0].geometry!.materials // Add new node to our empty scene emptyScene.rootNode.addChildNode(node) let fileManager = FileManager.default let appSupportDirectory = try! fileManager.url(for: .applicationSupportDirectory, in: .userDomainMask, appropriateFor: nil, create: true) let permanentUrl = appSupportDirectory.appendingPathComponent("converted.usdz") if emptyScene.write(to: permanentUrl, delegate: nil) { // We exported, now load and display return permanentUrl } } return nil }() var body: some View { VStack { RealityView { content in // Add the initial RealityKit content if let scene = try? await Entity(contentsOf: modelUrl!) { // Displays middle and bottom models content.add(scene) } if let scene2 = try? await Entity(named: "JetScene", in: realityKitContentBundle) { // Displays top model using premade scene and exported as USDA. content.add(scene2) } } update: { content in // Update the RealityKit content when SwiftUI state changes if let scene = content.entities.first { let uniformScale: Float = enlarge ? 1.4 : 1.0 scene.transform.scale = [uniformScale, uniformScale, uniformScale] } } .gesture(TapGesture().targetedToAnyEntity().onEnded { _ in enlarge.toggle() }) VStack (spacing: 12) { Toggle("Enlarge RealityView Content", isOn: $enlarge) .font(.title) Toggle("Show ImmersiveSpace", isOn: $showImmersiveSpace) .font(.title) } .frame(width: 360) .padding(36) .glassBackgroundEffect() } .onChange(of: showImmersiveSpace) { _, newValue in Task { if newValue { switch await openImmersiveSpace(id: "ImmersiveSpace") { case .opened: immersiveSpaceIsShown = true case .error, .userCancelled: fallthrough @unknown default: immersiveSpaceIsShown = false showImmersiveSpace = false } } else if immersiveSpaceIsShown { await dismissImmersiveSpace() immersiveSpaceIsShown = false } } } } } #Preview(windowStyle: .volumetric) { ContentView() } To test this even further, I exported the generated USDZ and opened in Reality Composer. The added model was still broken while the test model in the scene was fine. This also further proved that import/export is fine and RealityKit is not doing something weird with the imported model. I am convinced this has to be something with the way I'm using ModelIO to import the STL file. Any help is appreciated. Thank you

Closure containing control flow statement cannot be used with result builder 'ViewBuilder' below was the code let feet = 9 let inch = 45 var body: some View { VStack{ for number in 1...10 { print(number) } }

I'm trying to import the USDZ file of a model with multiple textures attached to each part of the model. When I preview the file by double-clicking on the USDZ, it views fine. However, when I import it into Reality Composer Pro, it only shows the pink striped model. I also get the message - "Multiple root level objects exist for HU_EVO_SPY-8.usdc". There are so many components of the model that binding each texture to each component will be very difficult to do manually. How can I fix the file such that when I import to Reality Composer Pro, textures are attached to the model?