It may be because AirPlay doesn't allow you to use a persistent key. You have to check the canProvidePersistableContentKey property on AVContentKeyRequest, and/or check if it's an instance of AVPersistableContentKeyRequest, and if not, fetch a new key from the key server instead of using a previously downloaded persistent key. In AVAssetResourceLoaderDelegate-land, this was accomplished by checking if AVAssetResourceLoadingRequest's allowedContentTypes property (an array) contained AVStreamingKeyDeliveryPersistentContentKeyType.
Also, try setting allowsExternalPlayback on your AVPlayer instance to NO. From my conversation with some AVFoundation engineers at WWDC 2018, this means that the audio will be streamed to the origin device, decrypted, re-encrypted, and re-streamed to the target device. I think this actually means you *can* use persistent keys for AirPlay. It also fixes an unrelated issue where old AppleTV hardware can't change playback speed.