Is ARKit's detecting 3D object reliable enough to use?

Question

Created Sep ’23

Replies 5

Boosts 0

Views 1.6k

Participants 2

Hi, I'm doing a research on AR using real world object as the anchor. For this I'm using ARKit's ability to scan and detect 3d object.

Here's what I found so far after scanning and testing object detection on many objects :

It works best on 8-10" object (basically objects you can place on your desk)
It works best on object that has many visual features/details (makes sense just like plane detection)

Although things seem to work well and exactly the behavior I need, I noticed issues in detecting when :

Different lighting setup, this means just directions of the light. I always try to maintain bright room light. But I noticed testing in the morning and in the evening sometimes if not most of the time will make detection harder/fails.
Different environment, this means simply moving the object from one place to another will make detection fails or harder(will take significant amount of time to detect it). -> this isn't scanning process, this is purely anchor detection from the same arobject file on the same real world object.

These two difficulties make me wonder if scanning and detecting 3d object will ever be reliable enough for real world case. For example you want to ship an AR app that contains the manual of your product where you can use AR app to detect and point the location/explanation of your product features.

Has anyone tried this before? Is your research show the same behavior as mine? Does using LIDAR will help in scanning and detection accuracy?

So far there doesn't seem to be any information on what actually ARKit does when scanning and detecting, maybe if anyone has more information I can learn on how to make better scan or what not.

Any help or information regarding this matter that any of you willing to share will be really appreciated

Thanks

Boost

Answer 1

JoonAhn OP

Sep ’23

We have developed a software library (FindSurface SDK) that processes 3D measurement points to accurately estimate the shape, size, position and orientation of the workpiece in real time.

The accuracy of the software is the highest according to ISO 10360-6 (accuracy better than 1 micrometer for the length unit and 1 microradian for the angle unit in a volume of 1 m^3), and the operating speed is more than 50 objects/sec. on iPhone 14 Pro (Max) with LiDAR 3D camera.

A series of 3D measurement points can be a point cloud (or a series of vertices in a mesh). Recognizable and measurable shapes are planes, spheres, cylinders, cones and tori.

For the working principle and demo of FindSurface, please search on web and find the following link:

GitHub CurvSurf/FindSurface
YouTube CurvSurf
FindSurface web demo.

FindSurface may not be what you search for.

If you have a question, please contact the email address listed on GitHub CurvSurf/FindSurface.

0

Answer 2

JoonAhn OP

Sep ’23

If the shape, size, location, and orientation of the surface of a real object can be accurately estimated by processing vertices of ARMeshAnchor in real time, various AR applications are possible. Accurate real-time calculations ensure that such real objects can serve as moving anchors for virtual objects.

Furthermore, if depth values for 576 points of LiDAR are provided, AR at another level is possible. Even in totally dark environments where the RGB images required for motion tracking cannot be obtained, however 576 points of LiDAR are obtained. By processing LiDAR's 576 such points in real time, you can pinpoint the shape, size, location, and orientation of a real-world object's surface front of you, even in the total darkness. 576 points of LiDAR must have no concern about privacy problem. But bring much positive uses.

0

Answer 3

JoonAhn OP

Sep ’23

By processing LiDAR's 576 points in real time, you can determine the shape, size, location, and orientation of a real-world object's surface front of you, even in the total darkness. Not only the objects but also you are allowed to move. Apple Vision Pro Night Vision.png

0

Answer 4

JoonAhn OP

Oct ’23

Night vision enables the visually impaired to see. In other words, it provides information about the shape, size, position and posture of objects in front of a visually impaired person. FindSurface can provide real-time information about this.

0

Answer 5

JoonAhn OP

Oct ’23

The FindSurface SDK software library has a binary file size of about 300 KB for Windows 64-bit DLL or 1 MB (due to the module name texts) for iOS/iPadOS/macOS/Linux/Android framework. It is a really tiny little runtime library that contains many algorithms for region growing, segmentation, model fitting (orthogonal distance fitting), optimization, geometry, curvature analysis, statistics and probability theory without any commercial libraries or ML/DL algorithms. It is a simple container with optimized abstract mathematical formulas and strategies.

0