AVAudioPlayer init very slow on iOS 17 Simulator

On Xcode 15 (released version), the iOS 17 Simulator will stall / lag as soon as an AVAudioPlayer is initialized.

let audioPlayer = try AVAudioPlayer(contentsOf: URL)
audioPlayer.volume = 1.0
audioPlayer.delegate = self
audioPlayer.prepareToPlay()

Typically you would not notice this in a music app for example, but it is especially noticable in games where multiple sounds are being played using multiple instances of AVAudioPlayer. The entire app slows down because of it.

It works fine on real devices and on iOS <=16 Simulators.

I noticed that the iOS 17 simulator immediately shows an error in the console:

66.051 HALC_ProxyIOContext.cpp:1.329 HALC_ProxyIOContext::IOWorkLoop: skipping cycle due to overload

and in the past iOS Simulators have been having trouble with audio.

Unfortunately I cannot find a solution. Anyone?