I'm using Xcode 11.5.
I'm using test plans.
I have an app that uses several local Swift Packages.
When I test the app, I'd like to run the tests for the packages.
In the current scheme I've tried putting a checkmark for "SomePackageTests" under the Test column in the Build action.
The test plan, "SomeTestPlan", seems to show up as its own target in the Build action. It has a disclosure icon to its left. When I expand, I see the test targets for the app and for all local packages. The expanded test targets don't have their own checkboxes. I've also tried putting a checkmark for "SomeTestPlan" itself under the Test column in the Build action.
When I edit the test plan, I see the test targets for the app and for all local packages. I've tried enabling the package tests there too.
The app tests (unit and UI) build and run successfully. For example, they show up in the test navigator. I can right-click on them, select "Run", and they build and run successfully.
The package tests are also listed in the test navigator. They build successfully—I see a "Build Succeeded" message pop up—but they don't run successfully. The error looks like this:
2020-05-26 11:03:33.947959-0400 xctest[55965:34366669] Failed to create a bundle instance representing '/Users/someuser/Library/Developer/Xcode/DerivedData/SomeApp-eecekbbecwouyeagzirvwewncpem/Build/Products/Alpha-Debug-iphonesimulator/SomePackageTests.xctest'. Check that the bundle exists on disk.
Program ended with exit code: 1
(Note that "Alpha-Debug" is the name of both a build configuration and a test plan. I assume the above is a reference to the test plan, but I could be wrong.)
I *think* the problem is that Xcode is looking in the wrong place—but only for the package tests, not the app tests.
It's correct that there's no directory at that location. There is, however, a directory by that name in this location:
/Users/someuser/Library/Developer/Xcode/DerivedData/SomeApp-eecekbbecwouyeagzirvwewncpem/Build/Products/Variant-UBSan/Alpha-Debug-iphonesimulator/SomePackageTests.xctest/
Here's the key difference.
The first path has ".../Products/Alpha-Debug...".
The second path has ".../Products/Variant-UBSan/Alpha-Debug...".
"UBSan" is the name of my default (and one-and-only) test configuration in the test plan.
So: it seems like building builds to a test configuration dir, but running looks in a non-test configuration dir. At least for packages.
Is this an Xcode bug, or did I misunderstand something?
Again, the goal is to run all of my package tests when I test my app.
PS I *think* this was working before I switched to test plans, but I'm not absolutely certain.
I was able to resolve this. As far as I can tell, it's an Xcode bug.
The short version is that Xcode seems to assume that at least one test configuration will use the shared (default) test configuration, unmodified. That assumption can of course be wrong; it's possible that no test configuration exactly matches the defaults. The mistake can prevent Xcode from finding the test target bundles for local Swift Packages.
The long version...
* A test plan has a shared (default) test configuration.
* A test plan has one or more actual test configurations.
* Xcode places build products in directories whose names are based on their differences from the shared (default) configuration.
* The build products for test configurations that don't have any meaningful differences are placed under the root, e.g.: "/Products/***/...".
* The build products for test configurations that do have meaningful differences are placed under subirectories whose names are derived from the differences, e.g.: "/Products/Variant-ASan-UBSan/***/...", "/Products/Variant-UBSan/***/...".
* Some part of Xcode assumes that at least one test configuration will match the shared defaults. In other words, some part of Xcode assumes that a test bundle will always exist under the root, in "/Products/***/...".
* In my case that wasn't true; my one and only test configuration enabled UBSan. That caused its build products to be placed in "/Products/Variant-UBSan/***/...". So, at least for my local Swift Packages, Xcode was looking in the wrong place: it was looking in the root rather than in a variant subdirectory under the root.
* The workaround was to create a test configuration that didn't differ from the default.
===
As an aside, I mentioned that the local Swift Package targets were listed in the scheme. I eventually realized that that was a hold over from my pre-test plan configuration; that's how I was getting Xcode to test all packages together with the main app before converting. With the test plan in place I no longer need them listed directly in the scheme. Unfortunately, removing them didn't fix my actual problem.