What are the new features of Object 3D tracking in ARKit 5?

Question

samuelmorais OP

Created Jul ’21

Replies 3

Boosts 0

Views 1.3k

Participants 2

Hi,

I am developing an AR app for quite a long time and one of our requirements is the 3D object tracking.

I would like to know if ARKit 5 is really more robust to detect and track the position of a 3D object as stated by the release notes.

Also I would like to know if there is a XCode project available to test the 3D object scanner (not the AR Capture, this one) like the previous versions of ARKit.

Thanks,

Samuel

Boost

Answer 1

Vision Pro Engineer OP

Apple

Jul ’21

I'm not sure if I grasped your question correctly. There are two distinct features you might be referring to:

There's ARKit Object Detection, which lets you detect 3D objects in the real world and expose their location by ARObjectAnchors. This API has been around since ARKit 2 and can be tried out in the Scanning and Detecting 3D Objects sample that you mentioned.
By contrast, there's Object Capture, a new feature introduced in RealityKit 2. Object Capture is a way to create 3D reconstructions from real-world objects by taking multiple photos. These aren't scans for detecting/tracking the object in ARKit; the purpose of this API is to generate 3D model files (for example in the usdz file format) and view them in ARQuickLook, RealityKit, or the renderer of your choice. To learn more about this API, I recommend watching the Create 3D Models with Object Capture session from WWDC`21. On the session page, you'll also find links to Xcode projects for an iOS app to take pictures of the object, and a macOS app to generate the 3D model out of these pictures.

Hope this helps and please don't hesitate to ask if you have further questions!

1

Answer 2

samuelmorais OP

Jul ’21

Thank you for you answer.

I am asking about the ARKit Object Detection.

In this page, it says that

" Additional improvements (...) 3D object detection is more robust, as objects are better recognized in complex environments. (...) "

I wanted to know, please:

By "more robust" means that more types of objects can be detected? (I know the sentence mentions complex environments, but what I need is to detect objects that has few features (without any logo / drawing). Do you think this kind of object can be detected in this new version?
Where can I download the new Scanning Xcode project, or am I going to use the same one that is available since ARkit 2?

Thanks.

0

Answer 3

Vision Pro Engineer OP

Apple

Jul ’21

Thank you for clarifying! On the page you referred, the section “Discover more ARKit features” intends to highlight a few key features of ARKit, not specific to this year’s release. The new features specific to ARKit 5 are listed in the top section, “More to Explore with ARKit 5”.

To address your questions specifically:

Objects don't necessarily have to show logos/drawings, but they will need to exhibit a certain amount of feature points in order to be detected. The scanning app in the sample project will warn you if the number of feature points is too low for best detection results. However you can try saving the reference object anyways and see if it gets detected. The app's threshold intends to provide an approximate guidance, but there's a chance it still works with fewer features. I recommend you just give it a try.
The existing Object Scanning app is still the recommended sample app to use.

0