Beginning audio playback from INPlayMediaIntent

I'm currently in the process of implementing support for INPlayMediaIntent shortcuts from Siri in an app that plays audio to users. The challenge I'm working through at the moment is the ideal order in which to handle and begin playback considering the user interaction flow with the Siri interface, as Siri causes an interruption to audio playback with AVAudioPlayer. Here's the current flow:
1. User triggers Siri and speaks an invocation phrase as they've selected.
2. My Intent extension is asked for a handler in

handler(for intent: INIntent)

3. My intent handler responds to

handle(intent: INPlayMediaIntent, completion: @escaping (INPlayMediaIntentResponse) -> Void)

with

INPlayMediaIntentResponse(code: .handleInApp, userActivity: nil)

4. My main app then implements

application(UIApplication, handle: INIntent, completionHandler: (INIntentResponse) -> Void)

in the AppDelegate, where the class that manages audio playback will be given the context of what to play, and begin playback.

5. The audio manager is observing

AVAudioSession.interruptionNotification

so it the selector gets invoked with an `InterruptionType` of `.began`, which I respond to by pausing playback.

6. At this point, the user either waits for the Siri interface to dismiss by itself (I believe this is inline with the Auto-Lock setting of the user's device, which means it could never happen if they have auto-lock set to "Never") or the user manually dismisses the view.

7. The selector for the `interruptionNotification` gets called again, this time with `.ended`, in which case I can resume playback.

The issue I have is between points 5 and 7 - my aim is to allow the user to have a totally handsfree experience in beginning audio playback, but it's very awkward for the user if they have to wait a large amount of time for Siri to be dismissed before audio playback starts, or they have to manually dismiss it (which is potentially unsafe if they are driving).

Is there any better way to work this flow so that we don't have to deal with such a large delay in playback?

Thanks!