Found WWDC talk: https://developer.apple.com/videos/play/wwdc2021/10047/?time=1384
So it's definitely not possible to programmatically change the mic mode. But in the case of my app I cannot even figure out how to enable the "mic mode" selection in the app control center. The video only vaguely says:
In order to use mic mode your app must adopt the Core Audio AUVoiceIO audio unit
I don't know what they mean by AUVoiceIO. I am using the normal AVSession and AudioComponentInstance.