Post

Replies

Boosts

Views

Activity

Reply to How on earth do you compress an MV-HEVC video file in-app without stripping the video of its 3D properties?
Thank you @gchiste, This is quite interesting. My intent with the MVHEVC compression properties is to be able to allow users to compress spatial videos (whilst preserving the stereoscopic/3D effect) in-app. Seeing as I do not own an Apple Vision Pro device myself, the simulator is all I have to work with. And given your claim that you are seeing support for the MVHEVC compression properties on a real device, I would like to push this update for my app (that includes this compression functionality for spatial videos), but I'm a bit unsure how to go about it, seeing as I'm unable to tell from my end using the simulator whether my compressVideo function (the one from my sample project) actually works as intended without crashing and whilst preserving the stereoscopic effect of the original uploaded MV-HEVC video. Any advice on how to go about this? Thanks again!
Mar ’24
Reply to How on earth do you compress an MV-HEVC video file in-app without stripping the video of its 3D properties?
Thanks for getting back @gchiste That really is unfortunate to hear. I'll go ahead and file a Feedback Assistant report. Just as a last ditch effort prior to wrapping up our discussion... is there any way you know of that will allow me to compress MV-HEVC videos in-app on visionOS that doesn't involve the use of these compression properties? Any frameworks, techniques, libraries etc?
Mar ’24
Reply to How on earth do you compress an MV-HEVC video file in-app without stripping the video of its 3D properties?
Accidentally posted my last reply as a comment... not sure it effects anything but I'm just going to re-post that same comment as a reply here just in case: Thank you so much for continuing to help me out with this @gchiste, it really is so beyond appreciated. I went ahead and created a sample project that reproduces the error and crashing that I'm experiencing when trying to compress an MV-HEVC video with my current MV-HEVC configuration settings. I have attached the link to my sample project's zip file below. My Sample Project (that reproduces the crashing error)
Mar ’24
Reply to How on earth do you compress an MV-HEVC video file in-app without stripping the video of its 3D properties?
Hey there @gchiste, thanks for getting back. I implemented the proper video settings to tell the asset writer to output an MV-HEVC video, but upon testing this implementation, by passing an MV-HEVC video through my compressVideo function, my app crashes with the below error: “Thread 3: "*** -[AVAssetWriterInput initWithMediaType:outputSettings:sourceFormatHint:] Compression property MVHEVCLeftAndRightViewIDs is not supported for video codec type hvc1"” Here’s what my compressVideo function looks like when the crashing occurs: let videoReaderSettings: [String: Any] = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32ARGB] let multiviewCompressionProperties: [String: Any] = [kVTCompressionPropertyKey_MVHEVCVideoLayerIDs as String: MVHEVCVideoLayerIDs, kVTCompressionPropertyKey_MVHEVCViewIDs as String: MVHEVCVideoLayerIDs, kVTCompressionPropertyKey_MVHEVCLeftAndRightViewIDs as String: MVHEVCVideoLayerIDs, kVTCompressionPropertyKey_HasLeftStereoEyeView as String: true, kVTCompressionPropertyKey_HasRightStereoEyeView as String: true ] let videoSettings: [String: Any] = [AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoHeightKey: videoTrack.naturalSize.height, AVVideoWidthKey: videoTrack.naturalSize.width, AVVideoCompressionPropertiesKey: multiviewCompressionProperties ] let assetReaderVideoOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: videoSettings) let assetReaderAudioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: nil) if assetReader.canAdd(assetReaderVideoOutput) { assetReader.add(assetReaderVideoOutput) } else { completion(.failure(NSError(domain: "VideoUploader", code: -4, userInfo: [NSLocalizedDescriptionKey: "Couldn't add video output reader"]))) return } if assetReader.canAdd(assetReaderAudioOutput) { assetReader.add(assetReaderAudioOutput) } else { completion(.failure(NSError(domain: "VideoUploader", code: -5, userInfo: [NSLocalizedDescriptionKey: "Couldn't add audio output reader"]))) return } let audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: nil) let videoInput = AVAssetWriterInput(mediaType: .video, outputSettings: videoSettings) videoInput.transform = videoTrack.preferredTransform I have also tried multiple different implementations to configure the output settings for MV-HEVC (such as the one below), but I am still thrown the same error and my app still crashes. let videoSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoHeightKey: videoTrack.naturalSize.height, AVVideoWidthKey: videoTrack.naturalSize.width, AVVideoCompressionPropertiesKey: [ AVVideoAverageBitRateKey: bitrate, kVTCompressionPropertyKey_MVHEVCVideoLayerIDs as String: [0, 1], kVTCompressionPropertyKey_MVHEVCViewIDs as String: [0, 1], kVTCompressionPropertyKey_MVHEVCLeftAndRightViewIDs as String: [0, 1], kVTCompressionPropertyKey_HasLeftStereoEyeView as String: true, kVTCompressionPropertyKey_HasRightStereoEyeView as String: true ] ] I have also tried simply removing the line of code “kVTCompressionPropertyKey_MVHEVCLeftAndRightViewIDs as String: MVHEVCVideoLayerIDs”, but upon doing so my app continues to crash and I receive a different error relating to the configuration (the one below): "Thread 7: "*** -[AVAssetWriterInput initWithMediaType:outputSettings:sourceFormatHint:] Compression property MVHEVCViewIDs is not supported for video codec type hvc1"” I should also note that I'm working on an M3 iMac with sonoma 14.3.1 as my OS, so I don’t think hardware should be the issue. I’ve also tested multiple different MV-HEVC video files but they all crash as well. Sorry for the long response, just figured I’d lay it all out to give out all the context I can. If there’s any changes I can make or anything you know of that will help fix this crashing error and allow me to compress MV-HEVC videos with the output also being MV-HEVC it would be greatly appreciated. Thanks.
Mar ’24
Reply to Video displaying black while audio plays in the background on Apple Vision Pro simulator
SOLVED: It took me literal months to figure this out but alas, there is actually a solution to this problem. If you ever use an AVPlayer in a visionOS app and find that your video is displaying as a black screen whilst the audio plays in the background, use some variant of this implementation below. The solution: create a custom view for your AVPlayer as is done below import SwiftUI import AVKit // Define a UIView subclass to host the AVPlayerLayer class PlayerView: UIView { private var playerLayer = AVPlayerLayer() // Initialize with an AVPlayer init(player: AVPlayer) { super.init(frame: .zero) playerLayer.player = player playerLayer.videoGravity = .resizeAspect // Adjust as needed layer.addSublayer(playerLayer) } required init?(coder: NSCoder) { fatalError("init(coder:) has not been implemented") } override func layoutSubviews() { super.layoutSubviews() playerLayer.frame = bounds } } // UIViewRepresentable wrapper for the PlayerView struct VideoPlayerView: UIViewRepresentable { var player: AVPlayer func makeUIView(context: Context) -> PlayerView { return PlayerView(player: player) } func updateUIView(_ uiView: PlayerView, context: Context) { // No need to update anything for now; the PlayerView handles resizing } } Then use this custom view in your program wherever needed like this: struct ContentView: View { var player: AVPlayer var body: some View { VideoPlayerView(player: player) } }
Feb ’24
Reply to Video displaying black while audio plays in the background on Apple Vision Pro simulator
Yea I suspected this too, so i ended up trying a few implementations without (frame: .zero), and with different constraints, but still experienced the same black screen issue. I've gone as far as testing empty makeUIView and updateUIView functions, and the issue prevailed. Seems as though any implementation of AVPlayer in a UIViewRepresentable will create this issue.
Jan ’24