synchronizing audio recording to playback

Question

Created Jun ’20

Replies 2

Boosts 0

Participants 2

I am playing back audio on an AVPlayer while recording audio with AVAcaptureSession. How can I synchronize the recording to the audio playback timeline. I am subtracting both inputLatency & outputLatency from AVAudioSession to the timestamp of the capture audio but the timing seems to be still be off. What other latencies do I need to compensate for?

Boost

Answer 1

Media Engineer OP

Apple

Jun ’20

Generally, you want to use the clocks / timebases of the player and the capture session to synchronize times.
Both objects already take care of latency, so you shouldn't need to consider that.

When both objects are playing / recording you can use code like this in captureOutput:didOutputSampleBuffer:fromConnection: to convert the sample buffer's presentation time from captureSession time to playback time.

Code Block CMClockRef clock = [captureSession masterClock];
CMTime captureTime = CMSampleBufferGetPresentationTime( sampleBuffer );
CMTime playerTime = CMSyncConvertTime( captureTime, clock, playerItem.timebase )

0

Answer 2

iami2 OP

Sep ’20

how do I synchronize playback to AVAudioRecorder start record time? Can I assume AVAudioRecorder uses the clock from CMClockGetHostTimeClock?

Code Block 		CMTime referenceTime = CMTimeAdd(CMClockGetTime(self.player.masterClock), CMTimeMakeWithSeconds(2,	NSEC_PER_SEC));	//start playback and record 2 seconds after.

[self.player setRate:1.0 time:kCMTimeZero atHostTime:referenceTime];
CMTime syncPTS = CMSyncConvertTime(referenceTime, self.player.masterClock, CMClockGetHostTimeClock());
[self.audioRecorder recordAtTime:CMTimeGetSeconds(syncPTS) forDuration:30]; //audioRecorder is an instance of AVAudioRecorder

0