Hello! Since Xcode 15 came out, I've been having this issue with UI tests where the test runner hangs for 60s when the keyboard is presented. In the test logs I get this kind of messages: 00:07.588 Wait for com.example.app to idle 01:07.642 App event loop idle notification not received, will attempt to continue. Interestingly, this only appears to be happening in CI environments (I've tried Xcode cloud and Circle CI). It works fine on my local machine. Everything is fine with Xcode 14, but I can't keep my CI environment stuck on Xcode 14 forever. I've tried disabling animations using UIView.setAnimationsEnabled(false), but it did not fix the problem. Here are the full logs of what it takes to fill out a text field: logs.txt

I'm trying to write a unit test for a SwiftData migration. In the teardown function I delete the SQLite container files, but then the underlying sqlite library complains. There must be a way to gracefully terminate the SwiftData container before I delete the files, but I don't see how. Simplying nil-ifying the references doesn't work. I don't see any obvious close functions, so I hope someone knows a non-obvious function. override func tearDownWithError() throws { // Cleanup resources // -- DOES NOT CLOSE UNDERLYING SQLITE ACCESS -- self.container = nil self.context = nil // Delete database do { try FileManager.default.removeItem(at: self.url) } catch { // Ignore file not found, report everything else. let nserror = error as NSError if nserror.domain != "NSCocoaErrorDomain" && nserror.code == 4 { throw error } } try? FileManager.default.removeItem(at: self.url.deletingPathExtension().appendingPathExtension("store-shm")) try? FileManager.default.removeItem(at: self.url.deletingPathExtension().appendingPathExtension("store-wal")) } I get these errors for .store, store-shm, and .store-wal: BUG IN CLIENT OF libsqlite3.dylib: database integrity compromised by API violation: vnode unlinked while in use: /Users/(ME)/Library/Developer/XCTestDevices/C52F4E12-EB4F-4639-9866-C3A7126155FA/data/Containers/Data/Application/B0EE90C6-B95D-4185-890D-6F20766B9B3B/tmp/test1.store invalidated open fd: 11 (0x11) If the worst comes to the worst, I'll work around it by using a differently-named container for each test, but as they're in tmp they'll get cleaned up for me eventually.

I am running xctests from command line on XCode 15.2 on iOS 17.2 simulators. As per WWDC2021 video on Memory issues, I am passing `-enablePerformanceTestsDiagnostics Yes' in following xcodebuild command. xcodebuild test -project project_name.xcodeproj -scheme test-scheme -destination 'platform=iOS Simulator,name=iPhone 15 Pro' -enablePerformanceTestsDiagnostics YES Final logs before test results: Tests-Runner(16873) MallocStackLogging: stack logs deleted from /private/tmp/stack-logs.16873.10148c000.Tests-Runner.M6gFw .xcresults is generated but no memgraph file is attached along with it.

Hi there, I'm trying to figure out an issue I'm seeing where the xcresult bundle isn't including the crash report after building and running with xcodebuild. I have a simple test app that has a single button that intentionally crashes and an XCUI test to run it. The UI Test also passes which can probably be explained by the lack of any asserts after clicking the button, but I assumed the app crashing would fail the test. The app itself is a Mac OS app with a single button called "Crash" that links to a method: - (IBAction)crash:(id)sender { @throw NSInternalInconsistencyException; } and a test method: XCUIApplication *app = [[XCUIApplication alloc] init]; [app launch]; XCUIElement *crashButton = [[XCUIApplication alloc] init].windows[@"Window"].buttons[@"Crash"]; XCTAssert([crashButton exists]); [crashButton click]; There is an ips file corresponding to the crash in ~/Library/Logs/DiagnosticReports so the crash gets written, but the test passes. After using xcresulttool to dump the Diagnostics folder from the xcresult bundle, I see these lines in the session log: 10:59:20.790 xcodebuild[23184:6634391] Handling Crash: MacAppTest (23189) main. libsystem_c.dylib: abort() called 10:59:20.792 MacAppTestUITests-Runner[23188:6635181] Process crashed with pid:23189 path:/Users/USER/Library/Developer/Xcode/DerivedData/MacAppTest-dxrjhasbuvyffhenipmdemzdpdht/Build/Products/Debug/MacAppTest.app/Contents/MacOS/MacAppTest bundleID:com.company.MacAppTest summary:main 10:59:20.792 MacAppTestUITests-Runner[23188:6635181] Ignoring crash for com.company.MacAppTest:23189 because we are not currently tracking it 10:59:21.371 MacAppTestUITests-Runner[23188:6634605] Returning result XCTWaiterResultCompleted for waiter <XCTWaiter: 0x6000025fd4d0> 10:59:21.371 MacAppTestUITests-Runner[23188:6634605] Wait <XCTWaiterWait: 0x600003e1b540> completed after 1.003s 10:59:21.371 MacAppTestUITests-Runner[23188:6634605] cleaning up delay The Ignoring crash for com.company.MacAppTest:23189 because we are not currently tracking it is what I suspect is the issue. How can I get the UITest to track the app so that when it crashes, the test fails? Some other lines from the logarchive file that may be helpful: 2024-02-14 10:59:20.612346 -0800 osanalyticshelper xpc log creation type 309 result success: /Users/<REDACTED>/Library/Logs/DiagnosticReports/MacAppTest-2024-02-14-105920.ips 2024-02-14 10:59:20.613030 -0800 testmanagerd Calling new file handler <__NSMallocBlock__: 0x6000031761f0> for MacAppTest-2024-02-14-105920.ips 2024-02-14 10:59:20.614240 -0800 ReportCrash client log create type 309 result success: /Users/<REDACTED>/Library/Logs/DiagnosticReports/MacAppTest-2024-02-14-105920.ips 2024-02-14 10:59:20.615417 -0800 testmanagerd Read crash report from /Users/<REDACTED>/Library/Logs/DiagnosticReports/MacAppTest-2024-02-14-105920.ips, got 17327 bytes of data Thanks!

I'm trying some sample with xcodebuild test command for i(Pad)OS simulators, figuring out some strange behavior on simulator initiation. 00:00:38 -------------------------- 00:00:38 --- Step: xcode_select --- 00:00:38 -------------------------- 00:00:38 Setting Xcode version to /Applications/Xcode/Xcode_15.3_beta_3.app for all build steps 00:00:38 Init simulators ... 00:00:39 $DEVELOPER_DIR 00:00:39 /Applications/Xcode/Xcode_15.3_beta_3.app/Contents/Developer ... 00:05:26 ▸ 2024-02-14 15:13:41.319 xcodebuild[12903:1082819] [MT] IDELaunchReport: 4a4f919bfba30580:4a4f919bfba30580: Finished with error: Simulator device failed to install the application. 00:05:26 ▸ Domain: IXErrorDomain 00:05:26 ▸ Code: 2 00:05:26 ▸ Failure Reason: Failed to locate promise. 00:05:26 ▸ User Info: { 00:05:26 ▸ FunctionName = "+[IXPlaceholder _placeholderForBundle:client:withParent:installType:metadata:placeholderType:mayBeDeltaPackage:error:]"; 00:05:26 ▸ IDERunOperationFailingWorker = IDELaunchiPhoneSimulatorLauncher; 00:05:26 ▸ SimCallingSelector = "installApplication:withOptions:error:"; 00:05:26 ▸ SourceFileLine = 803; 00:05:26 ▸ } 00:05:26 ▸ -- 00:05:26 ▸ Failed to set icon promise for com.example.myapp 00:05:26 ▸ Domain: IXErrorDomain 00:05:26 ▸ Code: 2 00:05:26 ▸ Failure Reason: Failed to create promise. 00:05:26 ▸ User Info: { 00:05:26 ▸ FunctionName = "+[IXPlaceholder _placeholderForBundle:client:withParent:installType:metadata:placeholderType:mayBeDeltaPackage:error:]"; 00:05:26 ▸ SourceFileLine = 803; 00:05:26 ▸ } 00:05:26 ▸ -- 00:05:26 ▸ Placeholder did not exist for UUID 2810E06A-4565-4966-BE98-AE4A6A96D916 when called for -[IXSClientConnection _remote_IXSPlaceholder:setIconPromiseUUID:completion:] by client CoreSimulatorBridge (pid 13037) 00:05:26 ▸ Domain: IXErrorDomain 00:05:26 ▸ Code: 3 00:05:26 ▸ Failure Reason: Failed to locate promise. 00:05:26 ▸ User Info: { 00:05:26 ▸ FunctionName = "-[IXSClientConnection _fetchPlaceholderForUUID:method:andRunWithPromise:error:]"; 00:05:26 ▸ SourceFileLine = 1945; 00:05:26 ▸ } 00:05:26 ▸ -- 00:05:26 ▸ 2024-02-14 15:13:41.323 xcodebuild[12903:1082819] [MT] IDELaunchReport: 4a4f919bfba30580:4a4f919bfba30580: com.apple.dt.IDERunOperationWorkerFinished { 00:05:26 ▸ "device_model" = "iPhone14,7"; 00:05:26 ▸ "device_osBuild" = "16.4 (20E247)"; 00:05:26 ▸ "device_platform" = "com.apple.platform.iphonesimulator"; 00:05:26 ▸ "dvt_coredevice_version" = "355.23"; 00:05:26 ▸ "dvt_mobiledevice_version" = "1643.100.52"; 00:05:26 ▸ "launchSession_schemeCommand" = Test; 00:05:26 ▸ "launchSession_state" = 1; 00:05:26 ▸ "launchSession_targetArch" = arm64; 00:05:26 ▸ "operation_duration_ms" = 108400; 00:05:26 ▸ "operation_errorCode" = 2; 00:05:26 ▸ "operation_errorDomain" = IXErrorDomain; 00:05:26 ▸ "operation_errorWorker" = IDELaunchiPhoneSimulatorLauncher; 00:05:26 ▸ "operation_name" = IDERunOperationWorkerGroup; 00:05:26 ▸ "param_debugger_attachToExtensions" = 0; 00:05:26 ▸ "param_debugger_attachToXPC" = 0; 00:05:26 ▸ "param_debugger_type" = 1; 00:05:26 ▸ "param_destination_isProxy" = 0; 00:05:26 ▸ "param_destination_platform" = "com.apple.platform.iphonesimulator"; 00:05:26 ▸ "param_diag_MainThreadChecker_stopOnIssue" = 0; 00:05:26 ▸ "param_diag_MallocStackLogging_enableDuringAttach" = 0; 00:05:26 ▸ "param_diag_MallocStackLogging_enableForXPC" = 0; 00:05:26 ▸ "param_diag_allowLocationSimulation" = 1; 00:05:26 ▸ "param_diag_checker_tpc_enable" = 0; 00:05:26 ▸ "param_diag_gpu_frameCapture_enable" = 3; 00:05:26 ▸ "param_diag_gpu_shaderValidation_enable" = 0; 00:05:26 ▸ "param_diag_gpu_validation_enable" = 1; 00:05:26 ▸ "param_diag_memoryGraphOnResourceException" = 0; 00:05:26 ▸ "param_diag_queueDebugging_enable" = 1; 00:05:26 ▸ "param_diag_runtimeProfile_generate" = 1; 00:05:26 ▸ "param_diag_sanitizer_asan_enable" = 0; 00:05:26 ▸ "param_diag_sanitizer_tsan_enable" = 0; 00:05:26 ▸ "param_diag_sanitizer_tsan_stopOnIssue" = 0; 00:05:26 ▸ "param_diag_sanitizer_ubsan_stopOnIssue" = 0; 00:05:26 ▸ "param_diag_showNonLocalizedStrings" = 0; 00:05:26 ▸ "param_diag_viewDebugging_enabled" = 0; 00:05:26 ▸ "param_diag_viewDebugging_insertDylibOnLaunch" = 0; 00:05:26 ▸ "param_install_style" = 2; 00:05:26 ▸ "param_launcher_UID" = 2; 00:05:26 ▸ "param_launcher_allowDeviceSensorReplayData" = 0; 00:05:26 ▸ "param_launcher_kind" = 0; 00:05:26 ▸ "param_launcher_style" = 0; 00:05:26 ▸ "param_launcher_substyle" = 0; 00:05:26 ▸ "param_runnable_appExtensionHostRunMode" = 0; 00:05:26 ▸ "param_runnable_productType" = "com.apple.product-type.application"; 00:05:26 ▸ "param_structuredConsoleMode" = 0; 00:05:26 ▸ "param_testing_launchedForTesting" = 1; 00:05:26 ▸ "param_testing_suppressSimulatorApp" = 1; 00:05:26 ▸ "param_testing_usingCLI" = 0; 00:05:26 ▸ "sdk_canonicalName" = "iphonesimulator17.4"; 00:05:26 ▸ "sdk_osVersion" = "17.4"; 00:05:26 ▸ "sdk_variant" = iphonesimulator; 00:05:26 ▸ } When I choose iPad 10th Generation, iPhone 14, 14 plus, 14 pro or 14 pro max devices listed on xcrun simctl list it fails with the log above. (doesn't matter what OS version is selected) Other than those sims it works (I checked it on iPhone SE 3rd gen, ipad mini 6g) Are those devices excluded from Xcode 15.3 Beta 3? Then why are those on the xcrun simctl list still? FYI: myapp took -Wl and -ld_classic for OTHER_LDFLAGS option in xcconfig after first app installation crash, still didn't make any changes.

I only see the output from the XCTest itself. Xcode: Version 15.2 (15C500b)

Hello, I'm having an issue with Unit testing on xCode 15.2 and macOS Sonoma. I'm trying to run unit testing and it hangs for a long time and then fails, previously I was able to run my unit testing on simulator with iOS 14, however after upgrading the xCode and the macOS I'm unable to use simulators with iOS 14 and my unit testing keeps failing. I have another device which is running macOS Ventura and I can run my unit testing on it without any problems using simulators with iOS 14 on xCode 15.0. Please note the issue is not with the xCode version because I tried downgrading and still have the same issue. I tried every solution i can find on this forum and others, I couldn't find anything that would help. I can't attach the dump log file directly since it's big so please use the link below https://drive.google.com/file/d/1v_gpCLhEuUdqFF6aCfrGislYWV8ZE8Qm/view?usp=share_link

Hi. I am measuring performances of methods in unit test on Xcode. Measuring performances on release build is important since you can know if a performance issue will be solved by compiler or not. However, when you specify its build as release, you cannot use @testable. Please tell me any manners. Thanks.

Xcode is not not picking up when I make changes to my unit test files when running tests. The only way is to clean the whole project and rebuild again. Any ideas why? I recently tried Xcode cloud and this behaviour started after removing that option

Since I start to wrote UITest, this keep happened with no waring, following image is the log, sensitive info has been replaced. However, once I run app, then run test again, UITest just work out, it seems like a wired but, and the method just a working around bug to me,

I am writing a SPM based project for MacOS. In this project? I need to access MacOS Keychain. I am write a swift test built by SPM testTarget(). I can see it generates a bundle ./.build/x86_64-apple-macosx/debug/MyProjectTests.xctest with an executable: % file ./.build/x86_64-apple-macosx/debug/MyProjectPackageTests.xctest/Contents/MacOS/MyProjectPackageTests ./.build/x86_64-apple-macosx/debug/MyProjectPackageTests.xctest/Contents/MacOS/MyProjectPackageTests: Mach-O 64-bit bundle x86_64 This bundle file cannot be executed. How can I execute its tests? I tried with xcodebuild test-without-building -xctestrun ./.build/x86_64-apple-macosx/debug/MyProjectPackageTests.xctest -destination 'platform=macOS' without any chance. Obviously the next question is can I 'simply' add entitlement to this bundle with codesign to fix my enttilement error. My error when running the test is A required entitlement isn't present.

In my iOS project, there is an infrequent crash related to virtual memory problem. Therefore, I plan to use UITest in combination with Product/Perform Action/Profile "TestCaseName" to conduct Game Performance-type testing. This allows the automatic testing to continuously operate until the profile stops recording upon a crash. This enables me to observe the various states of the program at the time of the crash. However, I have found that the UITest using Profile is highly unstable. The UITestCase often terminates unexpectedly during execution, leading to failed tests (Instruments is still working). Sometimes, the app is terminated immediately after startup. It seems that the use of sleep() in the code can easily cause interruption issues, which do not occur during normal UI testing. I am wondering if anyone has experience using Profile for UITest and whether they have encountered the issues I described." Working Environment: XCode14.3.1, iPhone Device iOS17.2

I have a CI pipeline, which runs unit tests and exports the code coverage. I need to run the sonar-scanner tool to export the report to the sonarqube server running on a server that can only be accessed over a VPN only. How can I do this?

I am directed from https://discuss.appium.io/t/create-multiple-instances-of-the-same-app/41266/14?u=lxnm, the context is that Appium max2 driver is implementing the activating of MacOS app using XCTest API methods. There are 2 ways to implement the activating of the app, using app path (calling initWithURL) and bundle id (calling initWithBundleIdentifier). This pull request shows how the XCTest methods are called when using the Mac2 driver, specifically in the file WebDriverAgentMac/WebDriverAgentLib/Routing/FBSession.m. The problem is that I am able to launch my MacOS app(it is a company app) using bundle id, but when I launch the app with app path, I receive XcodeBuild errors: [WebDriverAgentMac] [xcodebuild] XCTExpectFailure: matcher accepted Assertion Failure: Failed to launch com.company.companyApp: You do not have permission to run the application “companyApp”. You don’t have permission. To view or change permissions, select the item in the Finder and choose File > Get Info. (Underlying Error: The operation couldn’t be completed. Operation not permitted) I followed this to enable R+W permissions to all users, but the same error is displayed.

I am currently working on a SwiftUI project and I am trying to write a UI test for a DatePicker. However, I am having trouble interacting with the DatePicker in the test. Here is the code for the DatePicker in my SwiftUI view: DatePicker("Date", selection: $date, displayedComponents: [.date]) .datePickerStyle(.graphical) .labelsHidden() .frame(width: 150) And here is the code for my UI test: func testSelectingDateChangesMainView() throws { let app = XCUIApplication() app.launch() let dateDatePicker: XCUIElement = app.windows["mainUI-AppWindow-1"].datePickers["Date"] dateDatePicker.click() let col = dateDatePicker.descendants(matching: .any) print("\(dateDatePicker.debugDescription), \(col.debugDescription)") } When I run the test, it seems like the DatePicker does not have any descendants, so I am unable to select a date. Does anyone have any suggestions on how I can interact with a .graphical style DatePicker in a UI test? Any help would be greatly appreciated. Thank you, OnlyForF1

Hi! I'm experimenting with Xcode performance testing. I'm specifically focusing on XCTMemoryMetric. I have a demo project with two performance tests that do some work wrapped with XCTestCase.measure. I can navigate to my Report navigator in Xcode and see the memory footprints of these two tests… but I only see how to view them individually: From this screen it looks like I have the ability to Export this benchmark to CSV. My question is what my options are for viewing all my XCTestCase.measure tests together. I see the Duration for both tests… but I don't see any way to view the memory benchmarks for all my tests in just one place: Ideally… I would also like to run these tests and produce one CSV file (or XML or JSON) report. I'm not opposed to using command line for this (if the option exists there) but Xcode also works for me. I'm also flexible whether or not these tests live in a Swift Package or directly in an Xcode project if that makes any difference. Any more advice about how to set up these performance tests? Thanks!

We recently updated to Xcode 15.1 and start using iOS 17.2 simulator and ran into a blocker issue with out UITests. Setting accessibility identifier on a UIButton with image no longer works. It seems iOS automatically set the label to the file name used for the button and ignore (overwrite) the id and value we set in code. iOS 17.0 with Xcode 15.1 still works. I spent 2 days on this and still cannot find a solution. Anyone had similar issue? I did file a feedback https://feedbackassistant.apple.com/feedback/13515676 Thanks!

Hi, I have created the applications (debug and runner) using xcodebuild. I have tried it both using Xcode 14.3.1 and 15.1. I used the xcodebuild build-for-testing... with -configuration set to Debug (I've also built the apps by omitting the -configuration altogether). The configurations are set correctly in Xcode, and I can run the automation from Xcode and running xcodebuild. I am using a different tool other than xcodebuild to run the tests (I'm using go-ios https://github.com/danielpaulus/go-ios). The issue I'm having is that right at the beginning of the test, I'm getting this error "Failed to retrieve test configuration from IDE". I know it's something to do with the two apps (debug and runner appp), because I've made comparison to a brand new application I had created using Xcode. I assume this is something to do with the build configurations we have in the app (Debug or Release), but they're all correctly set in Xcode, and I can also run the tests (albeit using the xctestrun file) without any problems. Do you know what could be causing this? In particular, what could cause the above error to be generated? Any help (however related or unrelated you think it could be) is greatly appreciated. Thanks in advance. Sep

I'm working on this project for the first time in a while, so I'm not sure if this issue started with Xcode 15 or what, but now when I run my UI tests I get the warning "“MyAppUITests-Runner” is from an unidentified developer and differs from previously opened versions. Are you sure you want to open it?" Code signing identity for the UITest target is set to "sign to run locally". Changing it to "Apple Development" doesn't help, and neither did a clean build. How do I need to configure it to eliminate the warning?

I've created a test plan that includes the below configurations. {"configurations": [{ ... "options": { "defaultTestExecutionTimeAllowance" : 60, "maximumTestExecutionTimeAllowance" : 60, "testTimeoutsEnabled" : true }], "defaultOptions": { "defaultTestExecutionTimeAllowance" : 60, "maximumTestExecutionTimeAllowance" : 60, "testTimeoutsEnabled" : true, .... } ... } If I write a test that intentionally will take longer than 60 seconds, executing the test locally fails as expected with the error message Test exceeded execution time allowance of 1 minute let app = XCUIApplication() app.launch() let missingButton = app.buttons.firstMatch XCTAssertTrue(missingButton.waitForExistence(timeout: 120), "When a timeout is > 60 seconds, the default max setting of 60 seconds is still enforced") However, when this test is run using this test plan in XCode Cloud, it reports that it took 124 seconds to run and there is no message about exceeding execution time allowance. How can the test step be configured in XCode Cloud to ensure that the execution time allowances are respected?