Hello everyone!
I work for a company where we have a relatively large XCUI Test suite. When we utilize the Xcode simulator for running the suite, everything works extremely well for Software Engineers - everyone is happy!
The problem comes when we attempt to run the suite using real devices from a Third Party Provider's cloud-based lab. The outcome is polar opposite of what we see on the simulator. For example - locally 100% pass, on the real devices ~80% pass.
A few examples of the failures would be:
- Application not able to find element randomly
- Delays in Call prompts being displayed
Although it is properly handled programatically and works on local device
- Application randomly becoming unresponsive
I wanted to open this up to get help on two things:
- Technical differences/benefits of running on Simulator vs. Device (any insight into why they're so inconsistent)
- See if others have faced this issue and have advice 🙂
Thanks in advance for your help!