kAudioUnitSubType_AUiPodTime vs kAudioUnitSubType_AUiPodTimeOther

Hi,

We recently learned about the kAudioUnitSubType_AUiPodTimeOther pitch algorithm on iOS and it's greatly helped us get away from the limitations of kAudioUnitSubType_AUiPodTime.


These limitations were supported playback rates between 0 and 2x, as well as a great amount of echo in playback speeds below 1x. Our requirements are to support playback rates between 0x and 3x with negligble echo and high audio quality (we have voice content).


Is it possible to share what the underlying algorithm is behind kAudioUnitSubType_AUiPodTime vs kAudioUnitSubType_AUiPodTimeOther and why does it make such a big difference?


Why isn't kAudioUnitSubType_AUiPodTimeOther documented better to indicate these benefits?


Thanks

Accepted Reply

There are two "iPodTime" stretchers on iOS and from the complexity / performance vs. quality trade off view of the world you can think of these two as "simple" and "not so simple".

kAudioUnitSubType_AUiPodTime (iOS Only) - Simple - Time domain algorithm. This AU can only perform time stretching. Lowest quality, very fast, suitable for speech. When selecting the kAudioQueueTimePitchAlgorithm_LowQualityZeroLatency Audio Queue algorithm or AVAudioTimePitchAlgorithmLowQualityZeroLatency audioTimePitchAlgorithm, this is the AU being used.

kAudioUnitSubType_AUiPodTimeOther (iOS & OS X) - Not So Simple - Time domain algorithm. Much better quality and more flexible than AUiPodTime, fast, suitable for speech but can also be applied to music. When selecting the kAudioQueueTimePitchAlgorithm_TimeDomain Audio Queue algorithm or AVAudioTimePitchAlgorithmTimeDomain audioTimePitchAlgorithm, this is the AU being used.

The other "Time" AU available on iOS and OS X is kAudioUnitSubType_NewTimePitch. NewTimePitch is a frequency domain algorithm like the OS X only TimePitch and can do both pitch shifting and time stretching. This AU provides the highest quality time stretching and pitch shifting but is also the most computationally expensive. When selecting the kAudioQueueTimePitchAlgorithm_Spectral Audio Queue algorithm or AVAudioTimePitchAlgorithmSpectral audioTimePitchAlgorithm, this is the AU being used.

As for algorithms, Core Audio engineering were nice enough to provide the following information:

  • iPodTime - overlapping windows. no attempt to is made to synchronize overlaps according to pitch or waveshape.
  • iPodTimeOther - algorithm is similar to Waveform Similarity Overlap Add (WSOLA).
  • Spectral - phase vocoder.

AUComponent.h, AVAudioProcessingSettings.h and AudioQueue.h contain comments that can be put together to get an idea of the capabilities of the “Time” Audio Units available on both iOS and OS X.

Regarding the lack of documentation, as always please file enhancement request bugs for the documentation that you would like us to provide.

Replies

There are two "iPodTime" stretchers on iOS and from the complexity / performance vs. quality trade off view of the world you can think of these two as "simple" and "not so simple".

kAudioUnitSubType_AUiPodTime (iOS Only) - Simple - Time domain algorithm. This AU can only perform time stretching. Lowest quality, very fast, suitable for speech. When selecting the kAudioQueueTimePitchAlgorithm_LowQualityZeroLatency Audio Queue algorithm or AVAudioTimePitchAlgorithmLowQualityZeroLatency audioTimePitchAlgorithm, this is the AU being used.

kAudioUnitSubType_AUiPodTimeOther (iOS & OS X) - Not So Simple - Time domain algorithm. Much better quality and more flexible than AUiPodTime, fast, suitable for speech but can also be applied to music. When selecting the kAudioQueueTimePitchAlgorithm_TimeDomain Audio Queue algorithm or AVAudioTimePitchAlgorithmTimeDomain audioTimePitchAlgorithm, this is the AU being used.

The other "Time" AU available on iOS and OS X is kAudioUnitSubType_NewTimePitch. NewTimePitch is a frequency domain algorithm like the OS X only TimePitch and can do both pitch shifting and time stretching. This AU provides the highest quality time stretching and pitch shifting but is also the most computationally expensive. When selecting the kAudioQueueTimePitchAlgorithm_Spectral Audio Queue algorithm or AVAudioTimePitchAlgorithmSpectral audioTimePitchAlgorithm, this is the AU being used.

As for algorithms, Core Audio engineering were nice enough to provide the following information:

  • iPodTime - overlapping windows. no attempt to is made to synchronize overlaps according to pitch or waveshape.
  • iPodTimeOther - algorithm is similar to Waveform Similarity Overlap Add (WSOLA).
  • Spectral - phase vocoder.

AUComponent.h, AVAudioProcessingSettings.h and AudioQueue.h contain comments that can be put together to get an idea of the capabilities of the “Time” Audio Units available on both iOS and OS X.

Regarding the lack of documentation, as always please file enhancement request bugs for the documentation that you would like us to provide.

Awesome thanks for your help.