Hi, I'm building a windowed game in visionOS which requires a gamepad. And I'm using a PS5 controller for this case. However, I found a few problems: When looking at the window close button, and press X in the gamepad, the window will be closed. This can happen accidentally when the user is playing. When press the PS button(the home button), I want my app to handle it, e.g. take the user to the title screen. But visionOS will always capture this input and opens the home screen(App Library). How to avoid these 2 issues? Or in general, how to make the gamepad input only available to my app but not the visionOS system?

I'm building a streaming app on visionOS that can play sound from audio buffers each frame. The audio format has a bitrate of 48000, and each buffer has 480 samples. I noticed when calling audioPlayerNode.scheduleBuffer(audioBuffer) The memory keeps increasing at the speed of 0.1MB per second And at around 4 minutes, the node seems to be full of buffers and had a hard reset, at which point, the audio is stopped temporary with a memory change. see attached screenshot. However, if I call audioPlayerNode.scheduleBuffer(audioBuffer, at: nil, options: .interrupts) The memory leak issue is gone, but the audio is broken (sounds like been shortened). Below is the full code snippet, anyone knows how to fix it? @Observable final class MyAudioPlayer { private var audioEngine: AVAudioEngine = .init() private var audioPlayerNode: AVAudioPlayerNode = .init() private var audioFormat: AVAudioFormat? init() { audioEngine.attach(audioPlayerNode) audioEngine.connect(audioPlayerNode, to: audioEngine.mainMixerNode, format: nil) try? AVAudioSession.sharedInstance().setCategory(.playback, mode: .default) try? AVAudioSession.sharedInstance().setActive(true) audioEngine.prepare() try? audioEngine.start() audioPlayerNode.play() } // more code... /// callback every frame private func audioFrameCallback_Non_Interleaved(buf: UnsafeMutablePointer<Float>?, samples: Int) { guard let buf, let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 48000, channels: 2, interleaved: false), let audioBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(samples)) else { return } audioBuffer.frameLength = AVAudioFrameCount(samples) if let data = audioBuffer.floatChannelData { for channel in 0 ..< Int(format.channelCount) { for frame in 0 ..< Int(audioBuffer.frameLength) { data[channel][frame] = buf[frame * Int(format.channelCount) + channel] } } } // memory leak here audioPlayerNode.scheduleBuffer(audioBuffer) } }

I'm building a streaming app on visionOS that can play sound from audio buffers each frame. The source audio buffer has 2 channels and is in a Float32 interleaved format. However, when setting up the AVAudioFormat with interleaved to true, the app will crash with a memory issue: AURemoteIO::IOThread (35): EXC_BAD_ACCESS (code=1, address=0x3) But if I set AVAudioFormat with interleaved to false, and manually set up the AVAudioPCMBuffer, it can play audio as expected. Could you please help me fix it? Below is the code snippet. @Observable final class MyAudioPlayer { private var audioEngine: AVAudioEngine = .init() private var audioPlayerNode: AVAudioPlayerNode = .init() private var audioFormat: AVAudioFormat? init() { audioEngine.attach(audioPlayerNode) audioEngine.connect(audioPlayerNode, to: audioEngine.mainMixerNode, format: nil) try? AVAudioSession.sharedInstance().setCategory(.playback, mode: .default) try? AVAudioSession.sharedInstance().setActive(true) audioEngine.prepare() try? audioEngine.start() audioPlayerNode.play() } // more code... /// This crashes private func audioFrameCallback_Interleaved(buf: UnsafeMutablePointer<Float>?, samples: Int) { guard let buf, let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 480000, channels: 2, interleaved: true), let audioBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(samples)) else { return } audioBuffer.frameLength = AVAudioFrameCount(samples) if let data = audioBuffer.floatChannelData?[0] { data.update(from: buf, count: samples * Int(format.channelCount)) } audioPlayerNode.scheduleBuffer(audioBuffer) } /// This works private func audioFrameCallback_Non_Interleaved(buf: UnsafeMutablePointer<Float>?, samples: Int) { guard let buf, let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 480000, channels: 2, interleaved: false), let audioBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(samples)) else { return } audioBuffer.frameLength = AVAudioFrameCount(samples) if let data = audioBuffer.floatChannelData { for channel in 0 ..< Int(format.channelCount) { for frame in 0 ..< Int(audioBuffer.frameLength) { data[channel][frame] = buf[frame * Int(format.channelCount) + channel] } } } audioPlayerNode.scheduleBuffer(audioBuffer) } }

Hi, When opening an ImmersiveSpace with the .mixed style, is it possible to keep the user's current selected system immersive environment? Currently, the system immersive environment will be dismissed. ImmersiveSpace(id: "some id") { SomeRealityView() } .immersionStyle(selection: .constant(.mixed), in: .mixed)

Hi, One of the great features introduced in WWDC24 is the Translation API. But unfortunately it's currently unavailable on visionOS. My question is, does Apple have any plan to support it on visionOS as well? If so, what's the ETA for this feature? I would really like to see it on visionOS, otherwise I'll have to pay Google to use their translation API.

Hi, My app allows users to share and view spatial photos. For viewing spatial photos, I'm using a plane in a RealityView that has a camera index switch material node, which takes the stereo images as the inputs. For sharing native spatial photos taken on the vision pro, prior to visionOS 2.0, I extract the stereo image pair and merge them into a single side-by-side image to upload to the app's backend. However, since visionOS 2.0 introduced generating spatial photos from normal photos, I've been seeing some unexpected behaviours in my app, while on the other hand, they can be viewed correctly in the system Photos app: Sometimes the extracted images have different size, the right image is smaller than the left image. See the first image in the google drive below, taken with iPhone 15 Pro. Even if the image pair have the same size, when viewed in my app, it has some artefacts, especially around the edge of objects which are closer to the camera. See the second image in the google drive below, taken with iPhone 11. Google drive link here: https://drive.google.com/drive/folders/1UTfpxvO3-ChqshwfyzY5E_KCgk8VgUaa I know that now Quicklook preview application can support viewing spatial photos now, but I would like to keep it the way I implemented in the app, for compatibility concerns. Below is a code snippet that deals with the extraction. Please point out the correct way to extract stereo image pair from a generated spatial photo. Happy to submit a code-level support request if more information is needed. // the data is from photos picker item let data = try await photo.loadTransferable(type: Data.self) let source = CGImageSourceCreateWithData(data as CFData, nil) let sbsImage = source.extractSpatialPhoto() extension CGImageSource { func extractSpatialPhoto() -> UIImage? { guard let leftCGImage = extractSpatialImage(at: 0), let rightCGImage = extractSpatialImage(at: 1) else { return nil } let leftImage = UIImage(ciImage: leftCGImage) let rightImage = UIImage(ciImage: rightCGImage) guard leftImage.size == rightImage.size else { return nil } // merge left + right let size = CGSize(width: leftImage.size.width * 2, height: leftImage.size.height) UIGraphicsBeginImageContextWithOptions(size, true, 1.0) leftImage.draw(in: CGRect(x: 0, y: 0, width: leftImage.size.width, height: leftImage.size.height)) rightImage.draw(in: CGRect(x: leftImage.size.width, y: 0, width: rightImage.size.width, height: rightImage.size.height)) let mergedImage = UIGraphicsGetImageFromCurrentImageContext() UIGraphicsEndImageContext() return mergedImage } // not sure if this actually works func extractSpatialImage(at index: Int) -> CIImage? { guard let cgImage = CGImageSourceCreateImageAtIndex(self, index, nil) else { return nil } var ciImage = CIImage(cgImage: cgImage) if let properties = CGImageSourceCopyPropertiesAtIndex(self, index, nil) as? [String: Any], let heifDictionary = properties[kCGImagePropertyHEIFDictionary as String] as? [String: Any], let extrinsics = heifDictionary[kIIOMetadata_CameraExtrinsicsKey as String] as? [String: Any], let position = extrinsics[kIIOCameraExtrinsics_Position as String] as? [Double] { // Default baseline is 64mm (0 for left camera, 0.064m for right camera) let standardBaseline = 0.064 // Check if it's the right image (should be at [0.064, 0, 0]) let isRightImage = (index == 1) let expectedPosition = isRightImage ? standardBaseline : 0.0 // Calculate the translation needed to align to standard baseline let positionDelta = position[0] - expectedPosition // Apply translation only if there's a mismatch in position if positionDelta != 0 { let transform = CGAffineTransform(translationX: CGFloat(positionDelta), y: 0) ciImage = ciImage.transformed(by: transform) } } return ciImage } }

Hi, Following the wwdc24 video - What’s new in Quick Look for visionOS, I've managed to open a 3D model using PreviewApplication by calling let previewItem = PreviewItem(url: modelURL, displayName: "Easter", editingMode: .disabled) _ = PreviewApplication.open(items: [previewItem]) However, the "Save to Downloads" option is aways there(see attached screenshot). As the models are user generated content, and I don't want the download option to be available to all users. How to disable it?