I am using two iPhone11 devices, having iOS v17.6.1 for automation testing using Seleniumv4.X-Appium v2.x.
Today I updated the mac to Sequoia15 version which updated the XCode to v16.
Just after this update, I could not able to run the previously running automation script. The console message is "org.openqa.selenium.SessionNotCreatedException:
Could not start a new session. Response code 500. Message: Unable to launch WebDriverAgent. Original error: xcodebuild failed with code 70. This usually indicates an issue with the local Xcode setup or WebDriverAgent project configuration or the driver-to-platform version mismatch."
XCTest
RSS for tagCreate and run unit tests, performance tests, and UI tests for your Xcode project using XCTest.
Posts under XCTest tag
145 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I tried to use XCTest to test my own project that uses EndpointSecurity, but when I created the esClient I got an error:ES_NEW_CLIENT_RESULT_ERR_NOT_PRIVILEGED, indicating that it was not root.
This makes it impossible for me to do coverage tests for the ESClient application. Is there any way I can implement this ESClient test? If so, how should I use it? The project is a macOS program, if I use gcov, but I find I can't get coverage. Using __gcov_flush will indicate that there is no symbol
#if !TARGET_IPHONE_SIMULATOR
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
setenv("GCOV_PREFIX", [documentsDirectory cStringUsingEncoding:NSUTF8StringEncoding], 1);
setenv("GCOV_PREFIX_STRIP", "13", 1);
#endif
extern void __gcov_flush(void);
__gcov_flush();
#endif
I am creating a iOS library project using Xcode 15 in objective C lang which has dependency XCTest framework and Objective C run time and having UI test cases which can be use used in various projects.
I am able to build and run unit test cases using Xcode fine but when try to run test using command line getting below issue
Command
env NSUnbufferedIO=YES xcodebuild clean test -workspace EnptyLibrary.xcworkspace -scheme EnptyLibrary -configuration Debug -destination 'platform=iOS Simulator,OS=latest,name=iPhone 15 Pro' -destination 'platform=iOS Simulator,OS=latest,name=iPad Pro (12.9-inch) (6th generation)' -testPlan EnptyLibrary -derivedDataPath build -disable-concurrent-destination-testing -retry-tests-on-failure=NO
Error log
** TEST SUCCEEDED **
2024-09-12 16:36:34.822 xcodebuild[8345:45388802] Writing error result bundle to /var/folders/12/rcxtcvjs08169cc_fzhyc8x40000gq/T/ResultBundle_2024-12-09_16-36-0034.xcresult
2024-09-12 16:36:34.832 xcodebuild[8345:45388802] [MT] DVTAssertions: ASSERTION FAILURE in IDEFrameworks/IDEFoundation/ProjectModel/ActionRecords/IDESchemeActionsInvocationRecord.m:458
Details: Assertion failed: !self.hasSaved
Object: <IDESchemeActionsInvocationRecord: 0x112c21ce0>
Method: -saveWithCompletionBlock:
Thread: <_NSMainThread: 0x600000e441c0>{number = 1, name = main}
Hints:
Backtrace:
0 -[DVTAssertionHandler handleFailureInMethod:object:fileName:lineNumber:assertionSignature:messageFormat:arguments:] (in DVTFoundation)
1 _DVTAssertionHandler (in DVTFoundation)
2 _DVTAssertionFailureHandler (in DVTFoundation)
3 _DVTAssertionWarningHandler (in DVTFoundation)
4 -[IDESchemeActionsInvocationRecord saveWithCompletionBlock:] (in IDEFoundation)
5 -[IDESchemeActionsInvocationRecord saveSynchronouslyWithError:] (in IDEFoundation)
6 -[Xcode3CommandLineBuildTool saveResultBundleForError:triageWithSchemeTask:error:] (in Xcode3Core)
7 -[Xcode3CommandLineBuildTool _printError:schemeTask:andFailWithCode:] (in Xcode3Core)
8 -[Xcode3CommandLineBuildTool _printErrorWithSchemeTaskUserInfo:] (in Xcode3Core)
9 -[Xcode3CommandLineBuildTool _buildWithTimingSection:] (in Xcode3Core)
10 -[Xcode3CommandLineBuildTool run] (in Xcode3Core)
11 XcodeBuildMain (in libxcodebuildLoader.dylib)
12 -[XcodebuildPreIDEHandler loadXcode3ProjectSupportAndRunXcode3CommandLineBuildToolWithArguments:] (in xcodebuild)
13 -[XcodebuildPreIDEHandler runWithArguments:] (in xcodebuild)
14 main (in xcodebuild)
15 start (in dyld)
zsh: abort env NSUnbufferedIO=YES xcodebuild clean test -workspace -scheme EnptyLibrary
When I run a XCTest test on an IOS Target device, the test runs but the simulator is not opened so I can follow the screens being displayed and the key strokes. And when I initiate a record for a test method, it just hangs and never runs or open a simulator.
Is anyone else experiencing these same issues running XCTest?
Is this legacy framework not working with XCode 16 by design?
Hello,
I'm unable to run any test cases in Xcode 16 with the iOS 18 simulator.
Steps to Reproduce:
Create a new iOS project in Xcode 16 using the default iOS app template with the testing option checked.
Run any test or UI test case by clicking the "Play" icon in the Test navigator.
Xcode gets stuck at the "Testing" state indefinitely.
Environment:
macOS 15 beta 8
Xcode 16 beta 6
iOS 18 beta 7 simulator
Hello!
When running tests on iOS 17, the build succeeds but does not run on iOS 16. It consistently gets stuck on 'Testing'.
This issue does not occur on iOS 17.
XCTest can not get ios 18 contacts permission page elements, can not click "Allow Full Access" button.
Is it a good idea to automate API calls used in my mobile APP in XCtests and check if response is in expected format. I want to test all the API calls used inside my app at one place rather than testing each feature which internally calls that API. This way I want to sure that my API's are working
Hi everyone,
I've got an issue with the iPhone simulator version 15 iOS 17.0. Every time a test attempts to take a photo, the app directs to the camera software and then this error appears. For my colleagues instead, they've been directed to the gallery. The only difference is that my machine is M1 while theirs is with Intel chip. I'm using MacOS 13.6.3, the same as them.
Does anyone have any idea why this could happen? Thank you in advance.
Dear Apple Developer
[Background]
We are developing a service running on test mode based on XCTest Framework to support streaming, remote and automation testing with iOS devices.
After we upgrade macOS Sonama and Xcode 15 version for upgrade XCTest for support iOS 17 around Jan-2024, we saw
We noticed that our running on test mode services stuck and hanging frequently after a period of time.
We have investigate and tried debug code and we found that main thread stuck and hanging when it call requesting snapshot of accessibility hierarchy for app. At the same time, we noticed that the application we were testing stuck and hanging too.
[Environment]
Application testing: Tiktok
MacOS: Sonoma 14.5
Xcode: Version 15.4 (15F31d)
Device test: iPhone 12 (iOS 14.8.1)
Log detail:
t = 1524.15s Wait for com.ss.iphone.ugc.Ame to idle
t = 1536.54s Find the Application 'com.ss.iphone.ugc.Ame'
t = 1536.54s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 1597.56s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1)
t = 1597.56s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 1658.58s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2)
t = 1658.58s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 1718.58s Collecting extra data to assist test failure triage
t = 1718.58s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 1778.59s Requesting snapshot of accessibility hierarchy for app with pid 509
2024-08-19 14:16:44.990260+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0
2024-08-19 14:16:44.990428+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: XCTPerformOnMainRunLoop work timed out after 60.0s ((null)) (null) 0 0
t = 1843.71s Find the Application 'com.ss.iphone.ugc.Ame'
t = 1843.71s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 1874.72s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1)
t = 1874.72s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 1940.76s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2)
t = 1940.77s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2005.80s Collecting extra data to assist test failure triage
t = 2005.80s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2066.58s Requesting snapshot of accessibility hierarchy for app with pid 509
2024-08-19 14:21:33.019582+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0
2024-08-19 14:21:33.019687+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: Unable to perform work on main run loop, process main thread busy for 30.0s ((null)) (null) 0 0
2024-08-19 14:21:33.020647+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0
2024-08-19 14:21:33.020722+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: No element snapshot found for Application 'com.ss.iphone.ugc.Ame'[0.00, 0.00] -> (200.0, 259.0) ((null)) (null) 0 0
2024-08-19 14:21:33.020801+0700 WebDriverAgentRunner-Runner[489:97809] *** Assertion failure in -[XCPointerEventPath initForTouchAtPoint:offset:], XCSynthesizedEventRecord.m:263
t = 2291.08s Find the Application 'com.ss.iphone.ugc.Ame'
t = 2291.08s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2352.10s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1)
t = 2352.10s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2413.11s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2)
t = 2413.11s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2473.12s Collecting extra data to assist test failure triage
t = 2473.12s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2533.13s Requesting snapshot of accessibility hierarchy for app with pid 509
2024-08-19 14:29:19.552744+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0
2024-08-19 14:29:19.553054+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: XCTPerformOnMainRunLoop work timed out after 60.0s ((null)) (null) 0 0
t = 2598.27s Find the Application 'com.ss.iphone.ugc.Ame'
t = 2598.28s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2659.29s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1)
t = 2659.29s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2720.31s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2)
t = 2720.32s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2780.33s Collecting extra data to assist test failure triage
t = 2780.33s Requesting snapshot of accessibility hierarchy for app with pid 509
t = 2840.34s Requesting snapshot of accessibility hierarchy for app with pid 509
2024-08-19 14:34:26.806686+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0
2024-08-19 14:34:26.806828+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: XCTPerformOnMainRunLoop work timed out after 60.0s ((null)) (null) 0 0
2024-08-19 14:34:26.808114+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0
2024-08-19 14:34:26.808205+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: No element snapshot found for Application 'com.ss.iphone.ugc.Ame'[0.00, 0.00] -> (200.0, 259.0) ((null)) (null) 0 0
2024-08-19 14:34:26.809241+0700 WebDriverAgentRunner-Runner[489:97809] *** Assertion failure in -[XCPointerEventPath initForTouchAtPoint:offset:], XCSynthesizedEventRecord.m:263
Platform and Version
iOS
Development environment: Xcode 16.0 beta 5 (16A5221g), macOS 14.6.1 (23G93)
Run-time configuration: iOS 18.0 beta 5 (22A5326g)
Description of Problem
Starting with iOS 18 SDK, test bundles containing a +load method that accesses UIScreen.mainScreen result in deadlock during test bundle injection.
Also filed as FB14703057
I'm looking for clarity on whether this behavior is considered a bug in iOS or whether we will need to change the implementation of our app.
Steps to Reproduce
Create a new iOS app project using Objective-C and including unit tests
Add the following code snippet to any .m file in the test target:
@interface Foo: NSObject
@end
@implementation Foo
+ (void)load {
UIScreen * const mainScreen = UIScreen.mainScreen;
NSLog(@"%@", mainScreen);
}
@end
Run the tests
Expected Behavior
As with iOS 17 & Xcode 15, the tests run to completion.
Actual Behavior
With iOS 18 & Xcode 16, deadlock during test bundle injection.
stack_trace.txt
with the same issue.
https://developer.apple.com/forums/thread/100501
I am running xucitest framework, for e2e testing.
And webdriver app is running background, so it could't get clipboard while my tested app in running in front.
I could't switch app to get clipboard, because it will refresh my app.
Is there any way to allow clipboard permission with webdriver ?
I know, for secure reason, apple forbidden app the ability to get clipboard data in background.
But in some case, for example, i use appium to e2e testing my real device. And I could't get clipboard from webdriver. Because it's running background!
So, it's there any way to get clipboard from webdriver? It bothers me.
let metrics: [XCTMetric] = [XCTClockMetric(), // to measure time
XCTCPUMetric(), // to measure cpu cycles
XCTStorageMetric(), // to measure storage consuming
XCTMemoryMetric(),
]
let measureOptions = XCTMeasureOptions.default
measureOptions.iterationCount = 1
measure(metrics: metrics) {
//App flow
}
I want to get values of XCTCPUMetric, XCTMemoryMetric, XCTStorageMetric etc in any variable so that if want to send it further somewhere I can do it.
Example -
// let cpuMetric = CPU measure object should be here & I can get each information from this object.
// let MemoryMetric = Memory measure object should be here & I can get each information from this object.
But It's not available in XCUITest. We can only able to find it in the TestResult file. Please suggest any code available to get each metric object & value in the XCUITest rather than the test result.
`
let metrics: [XCTMetric] = [XCTClockMetric(), // to measure time
XCTCPUMetric(), // to measure cpu cycles
XCTStorageMetric(), // to measure storage consuming
XCTMemoryMetric(),
]
let measureOptions = XCTMeasureOptions.default
measureOptions.iterationCount = 1
measure(metrics: metrics) {
//App flow
}
`
I want to get values of XCTCPUMetric, XCTMemoryMetric, XCTStorageMetric etc in any variable so that if want to send it further somewhere I can do it.
Example -
// let cpuMetric = CPU measure object should be here & I can get each information from this object.
// let MemoryMetric = Memory measure object should be here & I can get each information from this object.
But It's not available in XCUITest. We can only able t find it in the TestResult file.
Please suggest any code available to get each metric object & value in the XCUITest rather than the test result.
Hi all,
I have a UI Automation Test suite with a large number of test cases, that typically takes me 7 hours to run. When I setup the workflow on XCode cloud, I received the following error - "Build has exceeded the maximum allotted time. Each build can run for up to 120 minutes. If a build has not yet finished at this time, all incomplete actions will be cancelled."
Is there any way to extend the build time?
What is a build? I have two actions in my workflow. Build Action (that took 3 minutes) and the test step that took 2 hour 4 minutes. Am not clear how the 120 minute window is calculated.
While using XCTest with Appium Inspector for iOS 18 on the "Select Contacts" screen, I expected XCTest to identify XCUIElement references for both the "Allow All Access" and "Select Contacts" buttons. However, neither element was found in the results and also there is no reference to any text in screen.
Feedback Assistant tag = FB14622782
I'm using the following code to launch a UI test that forces a specific app language and moves a slider.
I noticed that when forcing right to left text direction (for Arabic or Hebrew), during the UI test the slider doesn't move.
The app content:
struct ContentView: View {
@State private var slider = 0.0
var body: some View {
Slider(value: $slider, in: 0.0...1.0)
}
}
The UI test:
final class problemUITests: XCTestCase {
func testExample() throws {
let app = XCUIApplication()
let locale = Locale(identifier: "ar")
app.launchArguments += ["-AppleLanguages", "(ar)", "-AppleLocale", "ar"]
if locale.language.characterDirection == .rightToLeft {
app.launchArguments += ["-NSForceRightToLeftWritingDirection", "YES", "-AppleTextDirection", "YES"]
}
app.launch()
app.sliders.element.adjust(toNormalizedSliderPosition: 0.3)
}
}
Am I missing something or is there a workaround?
XCUIElement has two methods named typeKey(_:modifierFlags:): the first one takes a String as an argument, the second one a XCUIKeyboardKey, which has constants like .downArrow. Now, even if the documentation says that both methods are "Available in macOS and in iPadOS 15 and later", when compiling the UI tests for iOS, the following line
app.typeKey(.downArrow, modifierFlags: [])
produces a compiler error
Type 'String' has no member 'downArrow'
Am I missing something or is there a workaround?
I'm using Xcode Cloud for release builds and unit tests, and it works fine. Recently, I added some XCUITests, but to run it the way I need I started to use this mock server. Basically, the idea is you define before each test what responses you want to get for specific requests. It works like a charm locally. But if I want to run it in Xcode Cloud, it looks like the server has failed to launch. The mock server is a separately launched binary file. I think this is because this permission I get when launch it locally.
So, my question is how to allow this kind of permission in Xcode Cloud to make my UITests works correctly?
You can find a detailed instruction how server is launched on a github page I provided.