I'm developing a macOS app that interacts with Microsoft Teams using the Accessibility API. I've noticed inconsistent behavior when querying UI elements, particularly for the mute button. My queries often fail, while system tools like VoiceOver can consistently access these elements (which are visible on the screen).
In some cases, it works well, but in others, the UI elements are not visible from my code. When I try Accessibility Inspector, it also initially fails to inspect. However, the Inspector seems to have some "magical" power that, when I run it or via AX audit, appears to refresh the AX tree, and then my code occasionally works as well.
Given that VoiceOver can consistently read the screen, I assume the issue is not with the Microsoft Teams app itself (assuming it's based on Electron/React). I am mentioning this, because when I interact with Zoom app, reading the mute status from the app's menu bar, its 100% working anytime.
- What would you recommend I try or explore to improve reliability?
- Can I refresh the apps' AX tree from my end from swift?
- Is that a bug in AX API or even in Microsoft Teams?
(have ready example and demo video, but it does not let me upload here)