Indicate Packet Loss With AVAudioConverter for OPUS Decoding

I'm using an AVAudioConverter object to decode an OPUS stream for VoIP. The decoding itself works well, however, whenever the stream stalls (no more audio packet is available to decode because of network instability) this can be heard in crackling / abrupt stop in decoded audio. OPUS can mitigate this by indicating packet loss by passing a null pointer in the C-library to
int opus_decode_float (OpusDecoder * st, const unsigned char * data, opus_int32 len, float * pcm, int frame_size, int decode_fec), see https://opus-codec.org/docs/opus_api-1.2/group__opus__decoder.html#ga9c554b8c0214e24733a299fe53bb3bd2. However, with AVAudioConverter using Swift I'm constructing an AVAudioCompressedBuffer like so:

        let compressedBuffer = AVAudioCompressedBuffer(
            format: VoiceEncoder.Constants.networkFormat,
            packetCapacity: 1,
            maximumPacketSize: data.count
        )

        compressedBuffer.byteLength = UInt32(data.count)
        compressedBuffer.packetCount = 1
        compressedBuffer.packetDescriptions!
            .pointee.mDataByteSize = UInt32(data.count)

        data.copyBytes(
            to: compressedBuffer.data
                .assumingMemoryBound(to: UInt8.self),
            count: data.count
        )

where data: Data contains the raw OPUS frame to be decoded.

How can I specify data loss in this context and cause the AVAudioConverter to output PCM data whenever no more input data is available?

More context: I'm specifying the audio format like this:

        static let frameSize: UInt32 = 960
        static let sampleRate: Float64 = 48000.0

        static var networkFormatStreamDescription = 
            AudioStreamBasicDescription(
                mSampleRate: sampleRate,
                mFormatID: kAudioFormatOpus,
                mFormatFlags: 0,
                mBytesPerPacket: 0,
                mFramesPerPacket: frameSize,
                mBytesPerFrame: 0,
                mChannelsPerFrame: 1,
                mBitsPerChannel: 0,
                mReserved: 0
        )

        static let networkFormat = 
            AVAudioFormat(
                streamDescription: 
                     &networkFormatStreamDescription
            )!

I've tried 1) setting byteLength and packetCount to zero and 2) returning nil but setting .haveData in the AVAudioConverterInputBlock I'm using with no success.