Is it available to build a 3D model with depthData?

Question

I've built an app with ARKit 3.5 previously.

With [configuration.sceneReconstruction = .mesh],

I put all meshAnchors to 3D models.

Do I able to filter this meshAnchors by confidence,

and add color data from camera feed?

Or with demo code with MetalKit, how could I able to convert point cloud into 3d model?

1.9k

Posted by

HeoJin

Reply

Add a Comment

Answer 1

Hello HeoJin!

Confidence information is exposed in ARDepthData's confidenceMap property as part of the new depth API in ARKit 4. You can use it to filter the pixels of the depth map provided in the sceneDepth property.

The scene depth API introduced with ARKit 3.5 leverages the same depth data, but ARKit already provides an optimized mesh with the ARMeshAnchors so that you don't have to deal with confidence.

If you want to color the mesh based on the camera feed, you could do so manually, for example by unprojecting the pixels of the camera image into 3D space and color the according mesh face with the pixel's color. However, keep in mind that ARMeshAnchors are constantly updated. So you might want to first scan the entire area you're interested in, then stop scene reconstruction, and do the coloring in a subsequent step.

Posted by

Frameworks Engineer

Add a Comment

Answer 2

Hello HeoJin!

Confidence information is exposed in ARDepthData's confidenceMap property as part of the new depth API in ARKit 4. You can use it to filter the pixels of the depth map provided in the sceneDepth property.

The scene depth API introduced with ARKit 3.5 leverages the same depth data, but ARKit already provides an optimized mesh with the ARMeshAnchors so that you don't have to deal with confidence.

If you want to color the mesh based on the camera feed, you could do so manually, for example by unprojecting the pixels of the camera image into 3D space and color the according mesh face with the pixel's color. However, keep in mind that ARMeshAnchors are constantly updated. So you might want to first scan the entire area you're interested in, then stop scene reconstruction, and do the coloring in a subsequent step.

Posted by

Frameworks Engineer

Add a Comment

Answer 3

Thanks for your reply.
I'm not really familiar with the concept unprojecting pixels of image into 3D space and coloring.
Could you explain some little more?

Posted by

HeoJin

Add a Comment

Answer 4

Also wondering this.

Posted by

long_xiao_ge

Add a Comment

Answer 5

Also wondering this.

Posted by

Dast1

Add a Comment

Is it available to build a 3D model with depthData?

Accepted Reply

Replies