What are the proper macOS Security & Privacy that should be granted for Xcode to work properly on CI with Jenkins and Fastlane?
We have intermittent build failures causing about half of our CI builds to randomly fail in unpredictable ways. (See below for examples of the kinds of failures we get on about half our builds.)
So my question is: other than installing Xcode, is there anything else we need to do in order to make sure that the system can be used to build software via Jenkins and Fastlane?
Like do we need to add Xcode to "Developer Tools" in System Preferences > Security & Privacy > Privacy tab? What else do we need to add to that tab for Jenkins and Fastlane not to have all these problems?
All our build nodes are Mac Pros with 32 GB RAM so I know that hardware and resources aren't the problem.
We couldn't get 11.3 to work on Catalina at all, and at WWDC Apple told us that it was because the new Xprotect was attacking the simulators and it was fixed in Xcode 11.4 and later.
However many of the log messages we're seeing for the failures with 11.5 are the same exact log messages we were seeing when 11.3 would fail on Catalina.
It reminds me of an immune disease where antibodies attack healthy tissue.
As well, our unit tests often run extremely slow causing test failures due to timeouts. There is no apparent reason for this either.
I don't know why these issues are intermittent. I don't know why they are happening.
Please help.
Examples of the build failures:
One log message I see a lot is, os_unix.c:43353: (2) open(/var/db/DetachedSignatures) - No such file or directory
builtin-RegisterExecutionPolicyException /Users/jenkins/jenkins_workspace/build/ios/fastlane/output/derivedData/Build/Intermediates.noindex/ArchiveIntermediates/Redacted/InstallationBuildProductsLocation/Applications/Redacted.app note: Execution policy exception registration failed and was skipped: Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted" (in target 'Redacted' from project 'Redacted')
Or we'll have SimulatorTrampoline get a SIGKILL in the middle of test runs, and the error will say operation never finished bootstrapping, no restart will be attempted. Underlying error: test runner exited before starting test execution.
Or it'll be: Redacted.app (26246) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying error: Crash: Redacted (26246): +[XCTWaiter handleStalledWaiter:]))
Sometimes it results in a dialog box showing up on the associated build node that says "Simulator quit unexpectedly. Click Reopen to open the application again." Clicking "show details" on that box led to a NSInternalInconsistency exception.
Or we'll get:
Aug 16 14:50:03 Mac-102 CoreSimulatorService[494] <Error>: Error Domain=com.apple.CoreSimulator.SimError Code=165 "Unable to lookup in current state: Shutting Down" UserInfo={NSLocalizedDescription=Unable to lookup in current state: Shutting Down}
Or we'll get:
Aug 16 14:49:57 Mac-102 com.apple.CoreSimulator.SimDevice.FE9C2901-932B-416E-8931-41C1A5FE09FC[42746] (com.apple.xpc.launchd.domain.pid.testmanagerd.42775): Failed to bootstrap path: path = /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.4.simruntime/Contents/Resources/RuntimeRoot/usr/lib/dyld_sim, error = 2: No such file or directory Aug 16 14:49:57 Mac-102 com.apple.CoreSimulator.SimDevice.FE9C2901-932B-416E-8931-41C1A5FE09FC[42746] (com.apple.xpc.launchd.domain.pid.LogArchiveCollector.43535): Failed to bootstrap path: path = /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.4.simruntime/Contents/Resources/RuntimeRoot/usr/lib/dyld_sim, error = 2: No such file or directory
Or we'll get: codesign cannot open file at line errors.
Or we'll get: [2020-08-16T21:54:41.468Z] Error Domain=IDEProvisioningErrorDomain Code=9 ""Redacted.app" requires a provisioning profile with the Associated Domains, iCloud, and Push Notifications features." UserInfo={IDEDistributionIssueSeverity=3, NSLocalizedDescription="Redacted.app" requires a provisioning profile with the Associated Domains, iCloud, and Push Notifications features., NSLocalizedRecoverySuggestion=Add a profile to the "provisioningProfiles" dictionary in your Export Options property list.}
We have intermittent build failures causing about half of our CI builds to randomly fail in unpredictable ways. (See below for examples of the kinds of failures we get on about half our builds.)
So my question is: other than installing Xcode, is there anything else we need to do in order to make sure that the system can be used to build software via Jenkins and Fastlane?
Like do we need to add Xcode to "Developer Tools" in System Preferences > Security & Privacy > Privacy tab? What else do we need to add to that tab for Jenkins and Fastlane not to have all these problems?
All our build nodes are Mac Pros with 32 GB RAM so I know that hardware and resources aren't the problem.
We couldn't get 11.3 to work on Catalina at all, and at WWDC Apple told us that it was because the new Xprotect was attacking the simulators and it was fixed in Xcode 11.4 and later.
However many of the log messages we're seeing for the failures with 11.5 are the same exact log messages we were seeing when 11.3 would fail on Catalina.
It reminds me of an immune disease where antibodies attack healthy tissue.
As well, our unit tests often run extremely slow causing test failures due to timeouts. There is no apparent reason for this either.
I don't know why these issues are intermittent. I don't know why they are happening.
Please help.
Examples of the build failures:
One log message I see a lot is, os_unix.c:43353: (2) open(/var/db/DetachedSignatures) - No such file or directory
builtin-RegisterExecutionPolicyException /Users/jenkins/jenkins_workspace/build/ios/fastlane/output/derivedData/Build/Intermediates.noindex/ArchiveIntermediates/Redacted/InstallationBuildProductsLocation/Applications/Redacted.app note: Execution policy exception registration failed and was skipped: Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted" (in target 'Redacted' from project 'Redacted')
Or we'll have SimulatorTrampoline get a SIGKILL in the middle of test runs, and the error will say operation never finished bootstrapping, no restart will be attempted. Underlying error: test runner exited before starting test execution.
Or it'll be: Redacted.app (26246) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying error: Crash: Redacted (26246): +[XCTWaiter handleStalledWaiter:]))
Sometimes it results in a dialog box showing up on the associated build node that says "Simulator quit unexpectedly. Click Reopen to open the application again." Clicking "show details" on that box led to a NSInternalInconsistency exception.
Or we'll get:
Aug 16 14:50:03 Mac-102 CoreSimulatorService[494] <Error>: Error Domain=com.apple.CoreSimulator.SimError Code=165 "Unable to lookup in current state: Shutting Down" UserInfo={NSLocalizedDescription=Unable to lookup in current state: Shutting Down}
Or we'll get:
Aug 16 14:49:57 Mac-102 com.apple.CoreSimulator.SimDevice.FE9C2901-932B-416E-8931-41C1A5FE09FC[42746] (com.apple.xpc.launchd.domain.pid.testmanagerd.42775): Failed to bootstrap path: path = /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.4.simruntime/Contents/Resources/RuntimeRoot/usr/lib/dyld_sim, error = 2: No such file or directory Aug 16 14:49:57 Mac-102 com.apple.CoreSimulator.SimDevice.FE9C2901-932B-416E-8931-41C1A5FE09FC[42746] (com.apple.xpc.launchd.domain.pid.LogArchiveCollector.43535): Failed to bootstrap path: path = /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.4.simruntime/Contents/Resources/RuntimeRoot/usr/lib/dyld_sim, error = 2: No such file or directory
Or we'll get: codesign cannot open file at line errors.
Or we'll get: [2020-08-16T21:54:41.468Z] Error Domain=IDEProvisioningErrorDomain Code=9 ""Redacted.app" requires a provisioning profile with the Associated Domains, iCloud, and Push Notifications features." UserInfo={IDEDistributionIssueSeverity=3, NSLocalizedDescription="Redacted.app" requires a provisioning profile with the Associated Domains, iCloud, and Push Notifications features., NSLocalizedRecoverySuggestion=Add a profile to the "provisioningProfiles" dictionary in your Export Options property list.}
