Setting the default output device using Core Audio stops working after using continuity with AirPods

Hi, made an app which is managing the sound input/output for the user and I'm facing an unexpected behavior which can be replicated using the Apple's HALLab tool too.

Initially the app is able to set the input/output AudioDevice and everything works as expected however if you switch from your Mac to your iPhone when using AirPods Pro and switch back again the app is no longer able to set the output device. There's no error, it simply switches back to AirPods immediately.

You can replicate the issue on the HALLab(an app provided by Apple as "additional tools for Xcode").

How to:

1. Open HALLab, put on your AirPods and play some media.
2. Try out changing the input and output source and study the expected behavior
3. Unlock your iPhone, play some media and wait for the AirPods Pro to switch to the iPhone.
4. Go back to your Mac and play some media and wait for AirPods to switch to Mac
5. Try changing the output source on HALLab, notice that the source immediately reverses back to AirPods, which is the unexpected behavior.

Changing the source from the System Settings keeps working as expected.

Any ideas on what's going on and how to handle that?

I'm on MacOS 15.1 using the following code to set the device:

private func setDevice(deviceID: AudioDeviceID, forType type: AudioDeviceType) throws {

    guard isDeviceAvailable(deviceID) else {
        throw AudioDeviceError.deviceNotAvailable
    }
    print("setting the device.")
    var propertyAddress = AudioObjectPropertyAddress(
        mSelector: type == .input ? kAudioHardwarePropertyDefaultInputDevice : kAudioHardwarePropertyDefaultOutputDevice,
        mScope: kAudioObjectPropertyScopeGlobal,
        mElement: kAudioObjectPropertyElementMain
    )

    let dataSize = UInt32(MemoryLayout<AudioDeviceID>.size)
    var mutableDeviceID = deviceID // Create a mutable copy
    let status = AudioObjectSetPropertyData(AudioObjectID(kAudioObjectSystemObject), &propertyAddress, 0, nil, dataSize, &mutableDeviceID)
    guard status == noErr else {
        throw AudioDeviceError.deviceNotSet(status: status)
    }
}