I'm working on some apps at the minute, with the intention of running a lot of automated UI tests. Xcode cloud looks great and has a lot of integrated features, but having to make sure I don't run too many hours so my subscription doesn't stop mid dev cycle, or getting a surprise large bill etc, are things I want to avoid. Even the cheapest paid plan for a year works out about the same cost as an M2 mac mini, which would probably be significantly faster than the cloud nodes. Github actions allow you to provide your own machines. But actions requires a lot more scripting, and so far they've been very slow to update Macos/Xcode versions (not sure how this will effect supplying own machine). My preference would be to deploy my own machine for Xcode cloud, similar to what we used to be able to do with Xcode server. I think this is currently impossible right? Is there any word that this might be an added feature in the near future?

To authenticate Apple Pay payment through Mobile Test Automation via Face ID and a side button click if there is a way to automate a process involving Apple Pay where authentication could be performed using a software-based solution accessible through automation. Is there any way to access the side click button programmatically, using XCTest?

device : iPhone 8 Plus version: iOS 16 xcode version: 15.0.1 mac os: 13.6.6 When XCTest start to automation running, it does not cover the device's whole window as the picture shows. And my test cases testify that it cannot inspect the element outside of the cover. Is there any solution to make it compatible?

Hi all, I ran into an XCUITest issue where my tests fail randomly with the message that my app "has not loaded accessibility" I run the tests in my scheme in random order and with every run, some other random test fails and a test that previously failed would later pass. So I know it's not my test as-such that is causing the problem. There seems to be a 60s wait for accessibility to load, and I have not found any information how I could extend the timeout or convince accessibility to load faster. I found some older posts, but no real solution was found there either. This completely blocks my test execution since I cannot get a single run in where the tests all pass (and I only have 10 tests in that scheme). I ran the scheme against an iOs sim and against a real iOS device and I get the same random accessibility errors on either execution platform. I've tried iOS17.2 & 17.4 and get the same behavior. Strange enough, the test actually keeps running (and technically passes all the checks and asserts in the actual test) after the accessibility error, but then XCTest marks it as failed because of the early accessibility error. Any ideas what I could try or what the reason could be? Thanks! -- I'm using XCode Version 15.3 (15E204a) and test iOS17.2 and iOS17.4 Test Case '-[MyScheme.MyTestClass testMyFunction]' started. t = 0.00s Start Test at 2024-04-19 01:32:09.225 t = 0.02s Set Up t = 0.02s Open com.myCompany.myApp t = 0.03s Launch com.myCompany.myApp t = 0.25s Wait for accessibility to load t = 60.30s Capturing diagnostic spindump /Users/some/path/goes/here/MyTestCase.swift:120: error: -[MyScheme.MyTestClass testMyFunction] : Application 'com.myCompany.myApp' has not loaded accessibility t = 60.34s Waiting 60.0s for "test" Button to exist t = 61.38s Checking `Expect predicate `exists == 1` for object "test" Button` t = 61.39s Checking existence of `"test" Button` ... more stuff happening in the test here ...

So I have a simple view for my SwiftUI application below: import SwiftUI struct ContentView: View { var body: some View { List { ForEach(0..<4) { index in RowView(index: index) .accessibilityIdentifier("row") } } } } struct RowView: View { let index: Int var body: some View { HStack { Image(systemName: "globe") .imageScale(.large) .foregroundStyle(.tint) Text("Hello, world!") Spacer() ButtonView(index: index) } .padding() } } struct ButtonView: View { let index: Int var body: some View { VStack { if index != 3 { Button(action: {}, label: { Text("Cancel") }) .accessibilityIdentifier("cancel_button") } Button(action: {}, label: { Text("Submit") }) .accessibilityIdentifier("submit_button") } } } My goal is to reference each cancel and submit button per row for a UI Test, which I have tried doing so here: let rowElements = app.tables.cells.matching(identifier: "row") for i in 0..<rowElements.count { let cancelButton = rowElements.element(boundBy: i).buttons["cancel_button"] let submitButton = rowElements.element(boundBy: i).buttons["submit_button"] XCTAssertTrue(cancelButton.exists, "Cancel button does not exist in row \(i)") XCTAssertTrue(submitButton.exists, "Submit button does not exist in row \(i)") } Even with the identifiers set like this my tests fail and say it cannot find the buttons labeled as they are with their specific identifiers or the rows. What is the correct way to reference elements if they are in a list or table for UI Testing? I'm specifically trying to use accessibility Identifiers to make my life easy.

This code works when I run it in the iOS Simulator with iOS 17.0.1: let passcodeInput = springboard.secureTextFields["Passcode field"] _ = passcodeInput.waitForExistence(timeout: 10) passcodeInput.tap() However if I run it on the iOS Simulator with iOS 17.4 I get this error: t = nans Checking existence of `"Passcode field" SecureTextField` t = nans Capturing debug information t = nans Requesting snapshot of accessibility hierarchy for app with pid 66943 t = nans Tap "Passcode field" SecureTextField t = nans Wait for com.apple.springboard to idle t = nans Find the "Passcode field" SecureTextField t = nans Find the "Passcode field" SecureTextField (retry 1) t = nans Find the "Passcode field" SecureTextField (retry 2) t = nans Requesting snapshot of accessibility hierarchy for app with pid 66943 <unknown>:0: error: PRCheckUITests : Failed to tap "Passcode field" SecureTextField: No matches found for Descendants matching type SecureTextField from input {( Application, pid: 66943, label: ' ' )} Did the hardcoded string "Passcode field" change for iOS 17.4? How can I access the passcode field through springboard in a test?

I have a CI setup where one job builds xctestrun files for tests and another job that uses tem to actually run tests. The setup worked fine with XCode 14 on macOS 13. Now after switching to macOS 14 and XCode 15.0.1 xcodebuild doesn't want to run tests anymore: xcodebuild -destination 'generic/platform=iOS' -derivedDataPath DerivedData_Temporary/ -resultBundlePath './fastlane/test_output/APPName-iOS.xcresult' -testPlan 'APPName-Unit-All' -xctestrun 'DerivedData_Temporary/Build/Products/APPName_APPName-Unit-All_iphonesimulator17.0-arm64-x86_64.xctestrun' test-without-building throws an error xcodebuild: error: Failed to build workspace iOS with scheme APPName-iOS.: Scheme “APPName-iOS” does not have an associated test plan named “APPName-iOS-Unit-All” Almost the same command for macOS works perfectly fine. What does this error even mean? And how to fix it?

Hello! I've been trying to automate tests using Appium/XCUITests in a React Native APP, but I'm finding many blockers. They are related to the amount of nested elements in the "DOM" in which the XCUItest can not go deeper to get all the elements we need. snapshotMaxDepth -> The XCUITest does not support more than 60 value, otherwise it returns the following error: Got response with status 404: {"value":{"error":"stale element reference","message":"The previously found element "Application 'xyz.***.***'" is not present in the current view anymore. Make sure the application UI has the expected state. Original error: Error kAXErrorIllegalArgument getting snapshot for element <AXUIElementRef 0x600003aaf750> {pid=95967} {uid=[ID:1 hash:0x0]}","traceback":"(\n\t0 CoreFoundation 0x00007fff20405604 __exceptionPreprocess + 242\n\t1 libobjc.A.dylib 0x00007fff201a4a45 objc_exception_throw + 48\n\t2 WebDriverAgentLib 0x000000010a3caa53 -[XCUIElement(FBUtilities) fb_takeSnapshot] + 723\n\t3 WebDriverAgentLib 0x000000010a3cad07 -[XCUIElement(FBUtilities) fb_snapshotWithAttributes:maxDepth:] + 183\n\t4 WebDriverAgentLib 0x000000010a37baea +[FBXPath writeXmlWithRootElement:indexPath:elementStore:includedAttributes:writer:] + 778\n\t5 WebDriverAgentLib 0x000000010a37b12c +[FBXPath xmlRep... But if I inform less than 60, the XCUITest is not able to get all the elements we need to automate: There are many threads about this, all of them the issue is in the XCUITest: https://github.com/appium/appium/issues/14825 https://discuss.appium.io/t/handling-staleelementreferenceexception/35095/11 https://github.com/appium/appium/issues/18085 https://discuss.appium.io/t/error-in-appium-desktop-refreshing-source-after-adding-snapshotmaxdepth-greater-than-62/34058 https://stackoverflow.com/questions/74235441/appium-cant-reach-elements-in-ios-source-tree-they-are-too-deep-works-fine-in Tested all possible solutions suggested in the threads, but I will have the issue.

I've a non-document based single window (mostly) SwiftUI app. In my observable app state class an undo manager is defined and changes are registered using that manager. This has not been a problem in the past. But now... 1st is a bug reported in FB13708788. I've replaced the macOS undo/redo with code that references the UndoManager I created. The menu does not reflect the current undo status. This is easily duplicated in a small app submitted with the feedback. 2nd issue is hit or miss. Sometimes selecting Undo via the menu or the keyboard invokes undo multiple times. This is rare when running the app, but it has happened. On the other hand undo is always invoked multiple times when running an XCUITests. A little logging shows the invocations are quite close together. Undo invoked at 2024-04-05 13:14:19.975570-0700 Undo invoked at 2024-04-05 13:14:19.975768-0700 Calls were only 198 microsec apart. The test makes a change, makes a second change, then tried to undo the second change. Both changes are undone. But not (usually) if I perform the exact steps by hand instead of as an XCTest. Any ideas on how I might track down the cause? Mac Studio M2 Max running 14.4.1.

I tried to unit test the method "FilterDataProvider.handleNewFlow" from the sample project https://developer.apple.com/documentation/networkextension/filtering_network_traffic I always get this below issue, Undefined symbols: Linker command failed with exit code 1 (use -v to see invocation) Could someone help with it?

Got an issue with XC UI Testing where I can't open documents without them being marked as locked (the Locked label is shown in the titlebar). The document in question is a testing asset and copied into the testing bundle for easy access. I've checked the file in bundle in Finder and the document is not marked as locked. I've also tried copying the document to a temp directory before opening it and it's still locked?! What's the trick for getting it to open without being locked?

I am trying to do the tests form the terminal and xcpretty the output. Here is the command: xcodebuild -workspace bimb-authenticate-ios.xcworkspace -scheme bimb-authenticate-ios-afx -configuration Debug -destination 'platform=iOS Simulator,name=iPhone 14 Pro,OS=16.4' -enableCodeCoverage YES -derivedDataPath Build/ clean build test | xcpretty -c "--report" "html" "--output" "../Project_report.html" However, the tests are never run. The simulator is not even opened. Here's the last items of the xcbuild logs: ... *compiling, linting and linking logs* 2024-03-27 21:42:37.151 xcodebuild[84540:2237180] [MT] IDETestOperationsObserverDebug: 264.553 elapsed -- Testing started completed. 2024-03-27 21:42:37.152 xcodebuild[84540:2237180] [MT] IDETestOperationsObserverDebug: 0.001 sec, +0.001 sec -- start 2024-03-27 21:42:37.152 xcodebuild[84540:2237180] [MT] IDETestOperationsObserverDebug: 264.554 sec, +264.553 sec -- end ▸ Test Succeeded Can anybody help me? Thanks.

I am currently using Xcode 15.2, but I am sure this issue exists on other versions too. If you have any packages made pre Xcode 14.3, i.e. ones without an automatically made test plan, then the following issue happens: Let's call the package in this example PackageX, and you have with this a corresponding test scheme, called PackageXTests. PackageXTests is a test scheme saved on disk, that is on the test scheme of PackageX, meaning that when you test the package directly, it will run the test scheme. Now, with test plans becoming the norm, we want to move away from having these test schemes on disk and instead align old packages with new and have each package's tests dictated by its own corresponding test plan, the same way that any newly made package would work. So we click 'convert to test plan', success we have a test plan, we delete the test scheme from the disk. All appears to be working fine. However, if you now swap branch and come back, the test plan will no longer be connected to that package. It will have reverted back to a test scheme. Which in git shows as a test scheme which it has added (i.e. the git diff has an uncommitted test scheme it has made). This is not a derived data issue, as I have deleted my derived data and it still occurs. Similarly with cleaning and building.

Since couple of weeks ago (probably after I've updated my macbook to Sonoma 14.3.1) my Xcode cannot load unit test. When opening Test Navigator all I see is: Loading Tests... and it stays forever in that state. This is completely new project and this is crazy. Only way I can start testing is if I use CMD+U to run all the tests which is my company's project take a lot of time. Also there's no option to click on test-diamond-shaped button to run individual tests. Also tried checking "Test Plans" but I have infinite progress views: Anyone has any idea how to fix this? I tried to remove Derived Data, to reinstall Xcode (and different versions), did "git clean -fxd" nothing helped. Also some function like: "Rename", "Edit all in scope" stopped working as well.

My project has UI tests with total length of ~50 minutes. I'm using Xcode Cloud for builds delivery, UI tests execution is the part of the workflow. When I was using Monterey as tests environment everything was fine, but now it's no longer available so i had to switch to Sonoma. Since this i wasn't able to run anything but a couple of separate UI tests, when I run all of them I'm having 2 major issues. Locally on my machine I have the same Xcode/MacOS/simulator versions and everything works perfectly, issues only appear on Xcode Cloud. Here are they: There is a huge time gap between build-for-testing is finished and test-without-building is started. The gap is more than 1 hour so the sum of build time, ui tests time and the gap results in more than 2 hours and tests execution fails with the timeout. Funny is when I try to run the a single UI test or just run unit tests this gap only lasts for like 2-6 minutes, but when I add more tests the gap goes huge and fails everything. Also I guess my company is paying for those gaps and runs of tests when they timeout because of it even tho it's not our fault. Second issue is that some UI tests just freeze in the meddle of execution. This happens when UI tests is trying to interact with text fields on modal view controllers, sometimes it happens during the text field lookup, sometimes - after it takes focus and test is trying to type things into it. This is cannot be dodged anyhow and it is not related to some specific code - I've intentionaly rewritten the whole view controller from UIKit to SwiftUI , changed the way I present it and the location of presentation. Looks like just having a text field on a modal view is enough for test to freeze. Have anyone experienced something like this? Found any solutions?

I have latest version Xcode-15.1 & Mac-os version Sonoma-14.3.1 - while running particularly UITest target Xcode not launching the simulator, even after build succeed & test's completed without any errors. I tried cleaning project, re-starting Xcode, restarting system(Mac) - no luck It would be helpful if I get any suggestion on resolving the issue.

Sometime recently our XCUITests started to hang after awhile and so I had filed this Feedback ticket. With the release of Xcode 15.3 I re-ran the test in the project I had attached to that Feedback ticket (which taps a "Go!" button which pushes a new view controller and then verifies the value of a UILabel) and the tests are running crazy slow on a device now running iOS 17.4. It has been half an hour and only 14 of 100 iterations have been run. Anyone else experience severe performance degradation in their XCUITests with Xcode 15.3?

Hello world, I'm doing unit testing with Swift. I need to create a test for the registration process of an application. When I register, a confirmation link is sent to the specified email. How can I create a test that allows me to access the email and click on the link to confirm the registration?

I use ProcessInfo.processInfo.environment["key"] to get test parameters, but all of a sudden I could not find the UI like below: In the screenshot above, the 2 environment variables were actually what I entered for the unit test target. I don't have any idea why they were moved there! Any similar problems?

Hi, I'm in the team maintaining company-wise CI for iOS build and test, which runs over 100 macs on the system with every stable versions of Xcode/simulators and the latest beta/rc. I've been terribly suffered from xcodebuild test failure with simulator destination on Xcode beta series since Xcode 14.x has changed its CoreSimulator behavior from static simruntime into simdisk-based dynamic runtime. (If you search for xcrun simctl runtime match set 'watchos9.1' 20S75 you can check what I have been fighting for.) At the end of numerous nightly overtimes I finally found out that ~/Library/Developer/CoreSimulator/RuntimeMap.plist is closely related to xcrun simctl runtime match command. If CoreSimulatorService relaunches when RuntimeMap.plist is corrupted (or missing), Chosen Runtime will be set as default SDK (pretty sure it is SDK version of iphoneos, not iphonesimulator) ex) $ sudo xcode-select -s /Applications/Xcode_15.2.app $ sed -i '' -e 's|21C62|21Cblabla|' ~/Library/Developer/CoreSimulator/RuntimeMap.plist $ sudo pkill -9 com.apple.CoreSimulator.CoreSimulatorService $ xcrun simctl runtime match list == Evaluation Results == appletvos17.2: SDK Version: 17.2 SDK Build: 21K354 Platform: com.apple.platform.appletvos Chosen Runtime: tvOS 17.2 (17.2 - 21K364) - com.apple.CoreSimulator.SimRuntime.tvOS-17-2 User Override: (null) Preferred: tvOS 17.2 (17.2 - 21K364) - com.apple.CoreSimulator.SimRuntime.tvOS-17-2 Default: 21K354 iphoneos17.2: SDK Version: 17.2 SDK Build: 21C52 Platform: com.apple.platform.iphoneos Chosen Runtime: 21C52 User Override: (null) Preferred: 21Cblabla Default: 21C52 watchos10.2: SDK Version: 10.2 SDK Build: 21S355 Platform: com.apple.platform.watchos Chosen Runtime: watchOS 10.2 (10.2 - 21S364) - com.apple.CoreSimulator.SimRuntime.watchOS-10-2 User Override: (null) Preferred: watchOS 10.2 (10.2 - 21S364) - com.apple.CoreSimulator.SimRuntime.watchOS-10-2 Default: 21S355 xros1.0: SDK Version: 1.0 SDK Build: 21N301 Platform: com.apple.platform.xros Chosen Runtime: visionOS 1.0 (1.0 - 21N305) - com.apple.CoreSimulator.SimRuntime.xrOS-1-0 User Override: (null) Preferred: visionOS 1.0 (1.0 - 21N305) - com.apple.CoreSimulator.SimRuntime.xrOS-1-0 Default: 21N301 The question is: Where exactly is the path or file that reflects this change? What file is Chosen Runtime of runtime match list tracking? When is the time to re-download and fix if it is corroded or not found? P.S. Thanks valexeev4 for letting me know where RuntimeMap plist is. https://developer.apple.com/forums/thread/738077?answerId=765617022#765617022