I have been able to successfully export the
ARMeshGeometry with a color texture as a
USDZ file (I've not succeeded in doing so as an
.obj; the
.obj ends up looking very peculiar and the colors do not appear as expected). My understanding of 3D modeling and working with 3D technologies is cursory at best, but I was pretty interested in understanding how to do this. There are a few things to consider, especially if your starting point is the
Visualizing and Interacting with a Reconstructed Scene sample project;
The sample project is showing the beautiful mesh with colors by leveraging the arView.debugOptions.insert(.showSceneUnderstanding) call. This shows the mesh, with colors, using a debug method, which is not publicly accessible, nor representative of any particular color scheme relevant for export.
As far as I can tell, the sample project leverages RealityKit for rendering AR content, which does not have a method to generate a mesh from the ARMeshGeometry and texture it, nor does it have a built-in way of exporting any sort of 3D model as something like SceneKit does.
My approach was to set up a new AR project leveraging the SceneKit content technology, rather than RealityKit. This then assumes that you;
Set up an ARWorldTrackingConfiguration with the configuration's sceneReconstruction property set to .meshWithClassification.
Set up your ARSCNViewDelegate to receive delegate calls related to your ARSCNView.
In my
ARSCNViewDelegate method, I have the following functions configured;
Code Block | func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? { |
|
| guard let meshAnchor = anchor as? ARMeshAnchor else { |
| return nil |
| } |
|
| let geometry = SCNGeometry(arGeometry: meshAnchor.geometry) |
| |
| let classification = meshAnchor.geometry.classificationOf(faceWithIndex: 0) |
|
| let defaultMaterial = SCNMaterial() |
| defaultMaterial.fillMode = .lines |
| defaultMaterial.diffuse.contents = colorizer.assignColor(to: meshAnchor.identifier, classification: classification) |
| geometry.materials = [defaultMaterial] |
|
| let node = SCNNode() |
| node.geometry = geometry |
| return node |
| } |
Code Block | func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) { |
| guard let meshAnchor = anchor as? ARMeshAnchor else { |
| return |
| } |
|
| let newGeometry = SCNGeometry(arGeometry: meshAnchor.geometry) |
| |
| let classification = meshAnchor.geometry.classificationOf(faceWithIndex: 0) |
|
| let defaultMaterial = SCNMaterial() |
| defaultMaterial.fillMode = .lines |
| defaultMaterial.diffuse.contents = colorizer.assignColor(to: meshAnchor.identifier, classification: classification) |
| newGeometry.materials = [defaultMaterial] |
|
| node.geometry = newGeometry |
| } |
In general, I am using the
ARMeshGeometry to create a
SCNGeometry, classifying the "type" of mesh this is (table, floor, door, etc.) and setting a color based on that, then returning (or updating) the relevant node. Once you are ready to export your scene to a model, you can leverage SceneKit's
write(to:options:delegate:progressHandler:) method. For example, if your
ARSCNView is a property called
arView, you could call;
Code Block | let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask) |
| let documentsDirectory = paths[0] |
| let filename = directory.appendingPathComponent("mesh.usdz") |
| self.arView.scene.write(to: filename, options: nil, delegate: nil, progressHandler: nil) |
The general idea is to recreate the
ARMeshGeometry using a
SCNGeometry, set texture colors using whatever methodology works for your needs, then save the scene. Per the above code samples, there are some extensions necessary to convert the
ARMeshGeometry to a
SCNGeometry, which I will share in a follow-up post, as I believe there is a length limit here.
