Hello,
I want to capture video from Vision Pro in the Vision OS app. I am referring to the (https://developer.apple.com/videos/play/wwdc2024/10139/) Apple video and their code. step like below
- import ARKit
- com.apple.developer.arkit.main-camera-access.allow = true in info.plist
- Do below code
func loadCameraFeed() async {
// Main Camera Feed Access Example
let formats = CameraVideoFormat.supportedVideoFormats(for: .main, cameraPositions:[.left])
let cameraFrameProvider = CameraFrameProvider()
var arKitSession = ARKitSession()
var pixelBuffer: CVPixelBuffer?
await arKitSession.queryAuthorization(for: [.cameraAccess])
do {
try await arKitSession.run([cameraFrameProvider])
} catch {
return
}
guard let cameraFrameUpdates =
cameraFrameProvider.cameraFrameUpdates(for: formats[0]) else {
return
}
print(cameraFrameUpdates)
for await cameraFrame in cameraFrameUpdates {
print(cameraFrame)
guard let mainCameraSample = cameraFrame.sample(for: .left) else {
continue
}
pixelBuffer = mainCameraSample.pixelBuffer
}
}
I want to convert "pixelBuffer" into video streaming and show it in a frame like iOS.
Please guide me on how to achieve my next step. I am blank after this code.
Hey @Sandip_Aura,
If you haven't already, please make sure that you have requested the associated entitlements. Enterprise APIs are eligible for business use only, and you can only distribute apps that you develop with the Enterprise APIs privately as proprietary in-house apps or custom apps using Apple Business Manager.
Once you have the entitlement, use the init(cvPixelBuffer:) initializer create a CIImage. Once you have an CIImage, use the createCGImage(_:from:) method on a CIContext instance to create a CGImage. From there you can create a UIImage and then pass it to your SwiftUI Image view.
let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
let context = CIContext(options: nil)
if let cgImage = context.createCGImage(ciImage, from: ciImage.extent) {
Image(uiImage: UIImage(cgImage: cgImage))
}
Let me know if you have additional questions,
Michael
