Post

Replies

Boosts

Views

Activity

Reply to Performance issues with `AVAssetWriter`
Some additional information: Regarding the audio AVAssetWriter.init(...) call taking 3 seconds: These are the recommendedVideoSettingsForAssetWriter settings I get: ["AVVideoCompressionPropertiesKey": { AllowFrameReordering = 1; AllowOpenGOP = 1; AverageBitRate = 47848572; BaseLayerFrameRate = 15; ExpectedFrameRate = 60; MaxAllowedFrameQP = 41; MaxKeyFrameIntervalDuration = 1; MinAllowedFrameQP = 15; MinimizeMemoryUsage = 1; Priority = 80; ProfileLevel = "HEVC_Main_AutoLevel"; RealTime = 1; RelaxAverageBitRateTarget = 1; }, "AVVideoWidthKey": 2160, "AVVideoCodecKey": hvc1, "AVVideoHeightKey": 3840] (video initializes in ~10ms) And these are the recommendedAudioSettingsForAssetWriter(..) I get: ["AVEncoderQualityForVBRKey": 91, "AVEncoderBitRatePerChannelKey": 96000, "AVEncoderBitRateStrategyKey": AVAudioBitRateStrategy_Variable, "AVFormatIDKey": 1633772320, "AVNumberOfChannelsKey": 1, "AVSampleRateKey": 44100] (audio initializes in ~1500-3000ms) Note that subsequent calls are faster. I am also changing the AVAudioSessionCategory while configuring the audio AVAssetWriter, not sure if that has anything to do with it. Regarding the AVAssetWriter.startWriting() call taking 3-5 seconds: I still have no idea why that takes so long.
Nov ’23
Reply to iOS: Recording from two AVCaptureSessions is out of sync
Oh my god, I finally found the issue. The Video Stabilization mode cinematicExtended actually caused a huge delay in the Video Pipeline because it keeps an internal buffer of frames to stabilize them. This just takes time. Apparently this takes up to 1 second on my modern iPhone 15 Pro. The way my RecordingSession was designed is that I just wrote the Video and Audio buffers to the session once I received them, and I used their presentation timestamps as the timestamp for the video. What I didn't account for here was that the presentation timestamp is not the same value as the current time, the frame might be older and we just got it after it has went through a bunch of processing (stabilization)! E.g. relative to the timer we see on screen - when I start recording, the time is 5:00, but the first frame that actually arrives at 5:00 still has a presentation timestamp of 3:86, because that's when it was captured from the Camera. The mistake I previously made here was that we wrote this Frame to the file - which I shouldn't do, it's from the past! This PR fixes that behaviour and only writes Frames to the file that have a presentation timestamp later than when we actually started recording, so the first frame that should've been written to the file was the one with the presentation timestamp 5:00, which might come in when the current time is already 6:86, so it's 1:86 seconds too late. Here's my PR with code which fixes the issue: https://github.com/mrousavy/react-native-vision-camera/pull/2206 What a funny bug, I was pulling my hairs out on this one.
Nov ’23