Hi,
I'm trying to display an STL model file in visionOS. I import the STL file using SceneKit's ModelIO extension, add it to an empty scene USDA and then export the finished scene into a temporary USDZ file. From there I load the USDZ file as an Entity and add it onto the content.
However, the model in the resulting USDZ file has no lighting and appears as an unlit solid. Please see the screenshot below:
Top one is created from directly importing a USDA scene with the model already added using Reality Composer through in an Entity and works as expected.
Middle one is created from importing the STL model as an MDLAsset using ModelIO, adding onto the empty scene, exporting as USDZ. Then importing USDZ into an Entity. This is what I want to be able to do and is broken.
Bottom one is just for me to debug the USDZ import/export. It was added to the empty scene using Reality Composer and works as expected, therefore the USDZ export/import is not broken as far as I can tell.
Full code:
import SwiftUI
import ARKit
import SceneKit.ModelIO
import RealityKit
import RealityKitContent
struct ContentView: View {
@State private var enlarge = false
@State private var showImmersiveSpace = false
@State private var immersiveSpaceIsShown = false
@Environment(\.openImmersiveSpace) var openImmersiveSpace
@Environment(\.dismissImmersiveSpace) var dismissImmersiveSpace
var modelUrl: URL? = {
if let url = Bundle.main.url(forResource: "Trent 900 STL", withExtension: "stl") {
let asset = MDLAsset(url: url)
asset.loadTextures()
let object = asset.object(at: 0) as! MDLMesh
let emptyScene = SCNScene(named: "EmptyScene.usda")!
let scene = SCNScene(mdlAsset: asset)
// Position node in scene and scale
let node = SCNNode(mdlObject: object)
node.position = SCNVector3(0.0, 0.1, 0.0)
node.scale = SCNVector3(0.02, 0.02, 0.02)
// Copy materials from the test model in the empty scene to our new object (doesn't really change anything)
node.geometry?.materials = emptyScene.rootNode.childNodes[0].childNodes[0].childNodes[0].childNodes[0].geometry!.materials
// Add new node to our empty scene
emptyScene.rootNode.addChildNode(node)
let fileManager = FileManager.default
let appSupportDirectory = try! fileManager.url(for: .applicationSupportDirectory, in: .userDomainMask, appropriateFor: nil, create: true)
let permanentUrl = appSupportDirectory.appendingPathComponent("converted.usdz")
if emptyScene.write(to: permanentUrl, delegate: nil) {
// We exported, now load and display
return permanentUrl
}
}
return nil
}()
var body: some View {
VStack {
RealityView { content in
// Add the initial RealityKit content
if let scene = try? await Entity(contentsOf: modelUrl!) {
// Displays middle and bottom models
content.add(scene)
}
if let scene2 = try? await Entity(named: "JetScene", in: realityKitContentBundle) {
// Displays top model using premade scene and exported as USDA.
content.add(scene2)
}
} update: { content in
// Update the RealityKit content when SwiftUI state changes
if let scene = content.entities.first {
let uniformScale: Float = enlarge ? 1.4 : 1.0
scene.transform.scale = [uniformScale, uniformScale, uniformScale]
}
}
.gesture(TapGesture().targetedToAnyEntity().onEnded { _ in
enlarge.toggle()
})
VStack (spacing: 12) {
Toggle("Enlarge RealityView Content", isOn: $enlarge)
.font(.title)
Toggle("Show ImmersiveSpace", isOn: $showImmersiveSpace)
.font(.title)
}
.frame(width: 360)
.padding(36)
.glassBackgroundEffect()
}
.onChange(of: showImmersiveSpace) { _, newValue in
Task {
if newValue {
switch await openImmersiveSpace(id: "ImmersiveSpace") {
case .opened:
immersiveSpaceIsShown = true
case .error, .userCancelled:
fallthrough
@unknown default:
immersiveSpaceIsShown = false
showImmersiveSpace = false
}
} else if immersiveSpaceIsShown {
await dismissImmersiveSpace()
immersiveSpaceIsShown = false
}
}
}
}
}
#Preview(windowStyle: .volumetric) {
ContentView()
}
To test this even further, I exported the generated USDZ and opened in Reality Composer. The added model was still broken while the test model in the scene was fine. This also further proved that import/export is fine and RealityKit is not doing something weird with the imported model.
I am convinced this has to be something with the way I'm using ModelIO to import the STL file.
Any help is appreciated. Thank you