Thanks for the response Joar,
Continuous-integration servers (especially those which run as "Infrastructure-as-a-service" models like Travis, Cloudbees, CircleCI, Sauce Labs) aren't usually trusted with full Xcode project files.
Right now we use the Instruments commandline tool to run tests on a developer build of the app. This way, the only artifact required for testing is a compiled app file (with developer permissions).
Wouldn't xcodebuild require the full source of the app, the entire project file?
Appium tests specifically live outside of a developer's app, and are communicated via http to the appium server. How we would like to handle this is by building a standard "appium xcuitest" which is essentially a web server that waits for http commands and dynamically runs tests on the device. We've already experimented with building an "xcuitest app" for any given iOS app, but the exact sequence to kick-off the testing process on the device is not currently accessible to us.
Not sure if you've heard of Appium. It's an open source testing framework for mobile. It uses the same protocol as Selenium, the industry standard for programmatic browser UI testing. Appium has by now, after about three years of active development, become a (if not The) leading test automation framework for mobile. Sauce Labs, Perfecto Mobile, Xamarin Test Cloud, and Amazon all support appium testing in their Test Clouds. Check out discuss.appium.io for a large community of QA developers and test automation engineers who use appium daily all over the world (large representations in the USA, India, China, Japan). There's even multiple Appium meetups 🙂
Anyways, I'd be really interested in opening up a dialog about testing on iOS. We frequently come up with lots of bugs and corner cases in iOS automation and can hopefully contribute helpful information and repro cases for bugs.
Migrating our current iOS driver from UIAutomation to XCUITest would be great if we can figure out how to run our standard XCUITests on users' apps.
P.S. Would you consider applying to talk or attend GTAC, the Google Test Automation Conference? It's been focusing heavily on Mobile Test Automation the past couple years.
