Background
Xcode Test Plans, which debuted in Xcode 11, feature the ability to designate an .xcappdata bundle to copy into the test host app's container before tests run. (Before Test Plans, you could do the same thing by designating an .xcappdata to use on a per-target basis by editing the Test area of a Scheme.)
The UI for designating the .xcappdata to use for a given test target within the .xctestplan editor in Xcode 13
Back in Xcode 5 and prior, whenever you'd run your tests on a simulator or device, Xcode would copy the .xcappdata over to the simulator or device before the host app is launched for each test run.
However Xcode 6 introduceda bug where the .xcappdata does not get copied over to the simulator. Ever since then, from Xcode 6 through 13, .xcappdata is only copied over when you're running tests on a physical iOS device.
Current Workaround
Oddly, although Xcode itself doesn't handle copying the .xcappdata to a simulator, the terminal command xcrun simctl provides the ability to install an .xcappdata bundle to a given simulator or erase a given app's container.
Based on this, the workaround we're using is that I wrote some custom fastlane build scripts for our current Jenkins-based CI system, which sets up each simulator ahead of running the tests. And for local builds in Xcode itself, we have a pre-test action that runs a script to install the .xcappdata to the simulator.
Of course, we shouldn't have to implement these kinds of workarounds—it should just work. Apple shouldn't leave a glaring bug like this in Xcode for 5 years without fixing it.
Question
In the WWDC talk, "Explore Xcode Cloud workflows," it is stated:
Xcode Cloud provides a recommended option, which is a collection of simulators with different screen sizes
This indicates that, in Xcode Cloud, tests run in simulators, not physical devices. Given that the above bug is still in Xcode 13, in Xcode Cloud, how are we supposed to run test plans that specify .xcappdata bundles for particular test targets? Without this bug being fixed, the simulators won't have the .xcappdata bundles loaded into them, and the tests will always fail.
Given that the scripts that install .xcappdata to a simulator have to know the name of the simulator and have write-access to the directories where the simulator containers are stored, I'm a bit concerned about how feasible it would be to use a workaround similar to the one we're currently using.
It seems to me it would be far preferable if Apple would simply fix this long-standing bug. But I am curious to hear what Xcode Cloud team has to say about it.
BTW I still have an open DTS ticket and FeedbackAssistant ticket on this issue with no resolution:
- DTS ticket #765312786
- Feedback Assistant issue: FB9070373 (from me)
- Feedback Assistant issue: FB9127508 (from Github user @imWildCat, aka Daohan Chong of Microsoft)
Other references:
- Apr. 2015 StackOverflow: Xcode is not loading the .xcappdata into the app
- Apr. 2015 Open Radar 20622011: Xcode 6.3: Cannot use xcappdata package in simulator
- Aug. 2015 Open Radar 20481684: Xcode never copies the application data specified in a schema to the simulator
- Oct. 2015 Open Radar 23336152: Xcode 7.1: Cannot use xcappdata package in simulator
- Nov. 2015 Open Radar 23500433: Custom application data is ignored when launching app in iOS simulator
Thanks.