Scene understanding missing from visionOS simulator?

Question

Created Jun ’23

Replies 11

Boosts 11

Views 3.4k

Participants 15

SceneReconstructionProvider.isSupported and PlaneDetectionProvider.isSupported both return false when running in the simulator (Xcode 15b2).

There is no mention of this in release notes. Seems that this makes any kind of AR apps that depend on scene understanding impossible to run in the sim.

For example, this code described in this article is not possible to run in simulator: https://developer.apple.com/documentation/visionos/incorporating-surroundings-in-an-immersive-experience

Am I missing something or is this really the current state of the sim?

Does this mean if we want to build mixed-immersion apps we need to wait to get access to Vision Pro hardware?

Boost

Answer 1

mike_at_helium OP

Jun ’23

Filed bug report as FB12457682

2

Answer 2

Cosmo Luei OP

Jun ’23

If that is not supported in simulator, that will be very inconvenient to develop AR immersive apps before Vision Pro available. Hope apple fix this asap...

2

Answer 3

mike_at_helium OP

Jul ’23

Still missing in Xcode 15b3(15A5195k)

1

Answer 4

AlexxShadow OP

Jul ’23

Still missing in Beta5.. This has completely blocked development for our team.

0

Answer 5

J0hn OP

Aug ’23

Also filed this. I've about run out of things I can develop and feel I need to begin coding interactions with planes and the persistence of world anchors. https://feedbackassistant.apple.com/feedback/12639395

Hope the simulator gets this and I don't have to wait several years to continue.

0

Answer 6

Puasonych OP

Aug ’23

Still missing in Beta 6.

0

Answer 7

Oliver Drobnik OP

Aug ’23

Also found this, and reported it as FB.

0

Answer 8

atbf OP

Sep ’23

As of Xcode 15 Beta 8, all existing ARSession DataProviders report as unsupported in the visionOS simulator. Filed this as FB13125675.

1

Answer 9

JoonAhn OP

Sep ’23

It is logical that scene understanding is missing in the visionOS simulator. Scene understanding requires sensor data about the physical environment. The visionOS simulator without physical sensors cannot have sensor data.

Additionally, it should be noted that AVP includes cameras and LiDAR, but the sensor data is not shared with developers for privacy reasons. What is made available to developers is horizontal and vertical plane information and mesh information (ARMeshAnchor) that ARKit generates by internally processing sensor data.

0

Answer 10

fportela OP

Sep ’23

Although what @JoonAhn says is correct, what the visionOS simulator needs to do is provide "simulated" plane information that matches the environments that are provided by Apple. This should not be too difficult, and would remove the requirement of having devs invest on expensive hardware in order to create apps for it.

2

Answer 11

jaydeepsaravaiya OP

Oct ’23

its XCode Version 15.1 Beta(15C5028h), visionOS Simulator Version 15.1 (1018) and it's still the same. No DataProvider other than WorldTrackingProvider works in simulator. It's kinda funny that mesh data/camera position is already known to simulator and can populate ARKit sensor data very easily (with even more accuracy than real world) but who knows it might be too hard /s.

1