Error running xcodebuild test-without-build -xctestrun on other machines.

I have generated a .xctestrun file using "build for Testing" on a MacOS 64 bit application. My tests are UI tests only. I can go to terminal and run


xcodebuild test-without-building -xctestrun Application_macosx10.14-x86_64.xctestrun -destination 'platform=MacOS,arch=x86_64'


It runs all the UI tests and works great.


The problem I'm having is using this same Application_macosx10.14-x86_64.xctestrun on a different machine. We'd like to be able to run these UI tests on a Mac 10.12 Sierra machine. I copied the .xctestrun and the directories with the test runner and applicaiton and tests to the Sierra machine. I installed Xcode 9.2 and the command line tools. I open terminal in the directory where the .xctestrun file is and run


xcodebuild test-without-building -xctestrun Application_macosx10.14-x86_64.xctestrun -destination 'platform=MacOS,arch=x86_64'


I get the following error:


User defaults from command line:

IDETestRunSpecificationPath = /Users/sysqa/Desktop/UI Tests Release/build/Application_macosx10.14-x86_64.xctestrun


xcodebuild: error: Failed to build workspace temporary with scheme Transient Testing.

Reason: Validation for test run specification failed: TestBundlePath should be provided; TestHostPath should be provided; DYLD_FRAMEWORK_PATH should be provided in testing environment variables; DYLD_LIBRARY_PATH should be provided in testing environment variables


It looks like XCode 9.2 supportsthe test-without-building and -xctestrun options but doesn't like the .xctestrun file becuase all the things it's asking for are in the Application_macosx10.14-x86_64.xctestrun. Is there some difference between the format of the .xctestrun file between XCode 9.2 and 10.2? Is it possible to run these tests on other machines using the same .xctestrun file?

Up vote post of pmoprhy Down vote post of pmoprhy
3.9k views
Posted by
pmoprhy
Reply
Add a Comment

Replies

I have the same issue. Have you managed to solve this?

Posted by
shunp0
Up vote reply of shunp0 Down vote reply of shunp0
Add a Comment

Has anyone found a solution for this, or at least knows what causes this?

I've been having the same error when running UITests using xctestplan and on tvOS simulator, in combination with Fastlane: I'm building test target and runner using this Fastlane command:

bundle exec fastlane build_for_tests SCHEME:MyScheme 'TEST_DEVICE:Apple TV 4K (3rd generation)'

which results in this xcodebuild command:

xcodebuild -scheme MyScheme -project ./MyProject.xcodeproj -derivedDataPath ./build/DerivedData -destination 'platform=tvOS Simulator,id=XXXXXXXX-XXXX-XXXX-XXXX-230D173EFEC8' build-for-testing

I then use the following Fastlane:

bundle exec fastlane run_test_plan SCHEME:MyScheme TEST_PLAN:MyTestPlan 'TEST_DEVICE:Apple TV 4K (3rd generation)' TVOS_PLATFORM:appletvsimulator TVOS_VERSION:16.4 ARCH:x86_64 TEST_WITHOUT_BUILDING:true

It generates the following:

xcodebuild -destination 'platform=tvOS Simulator,id=XXXXXXXX-165C-4A5F-XXXX-230D173EFEC8' -derivedDataPath ./build/DerivedData -resultBundlePath './fastlane/test_output/UITests.xcresult' -testPlan 'MyTestPlan' -xctestrun './build/DerivedData/Build/PATH_TO_TESTRUN/MyTestRun.xctestrun' test-without-building

Using Xcode 15.3, Sonoma 14.3.1

Thanks for any help / context (especially about that Transient Testing) :)

Posted by
Sepsky
Up vote reply of Sepsky Down vote reply of Sepsky
Add a Comment

Hi again, one thing I need to add is that we don't actually get a reason for this, just the error and an error 64 code. Thanks

Posted by
Sepsky
Up vote reply of Sepsky Down vote reply of Sepsky
Add a Comment