Will it be possible to get direct access to all the different cameras of Vision Pro via AVFoundation or indirectly via other frameworks?
Which sensors (inner and outer color & IR cameras, LiDAR, TrueDepth camera) will developers have access to on visionOS?
Hi @rd1, AVFoundation is available on visionOS, but apps will not have access to cameras. Check out the session Enhance your iPad and iPhone apps for the Shared Space for more information about how existing iPadOS and iOS apps can interact with the hardware:
However, many of these cameras are not available for app use. It is critical to use discovery sessions to detect which cameras and microphones are available for use. To ensure apps have an excellent capture experience, use AVCaptureDevice discovery session to confirm hardware availability.
Thank you, @Developer Tools Engineer. It seems that not even ARKit will let developers access any sensor data: https://developer.apple.com/videos/play/wwdc2023/10082?time=262
In the WWDC video you shared, it was mentioned that only a "composite .front camera" will be abailable via AVCapture DiscoverySession. May I ask what is meant by "composite camera"? Does it mean that visionOS will provide stream of RGB-only frames captured by a virtual camera, which is created by internally combining data from several hardware cameras?
Or is the meaning of "composite" meant to be "multiple modalities" (i.e. AVFoundation providing access to RGB and depth output via a virtual composite camera)?
Is AVFoundation going to be the only Framework on visionOS capable of providing access to (composite) camera data, as suggested by the video you shared?
I found this app and I wonder how this is possible
@slzrafael you can access the cameras from the AVP if you run the apps as iPad apps in 2D in the Shared Space, just not building it from a purely for VisionOS since AVCaptureDevice.DiscoverySession is unavailable in VisionOS. If you call the front camera the device will automatically call the PersonaCamera into view
let deviceDiscoverySession = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInWideAngleCamera], mediaType: .video, position: .unspecified)
for device in deviceDiscoverySession.devices {
if device.position == .front {
userFacingCamera = device
}
}
currentDevice = userFacingCamera