@KTRosenberg I am also very concerned about the lack of ability to sense/interact with the world around you. I seems you will only get what xrOS decides to give you, which are currently simple things like walls, floors, etc... As you've no doubt noticed in the sessions there are repeated (and valid) references to user privacy. The downside is it seems like we are beholden to what xrOS shares with us, we can't implement any novel object detection of our own. This is of course v1, but it is vital to be able to sense/interact with the world around us for the ar vision to become a reality.
jroome
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution
Last seen
User for
Post
Replies
Boosts
Views
Activity
@KTRosenberg From looking at the SDK it looks like SceneReconstructionProvider (https://developer.apple.com/documentation/arkit/scenereconstructionprovider) will be able to get you the raw detected geometry MeshAnchor (https://developer.apple.com/documentation/arkit/meshanchor).
This doesn't solve the issue of pixel data access, but gives an idea of the lowest level scene understanding you'll be able to get (so far).