Create and run unit tests, performance tests, and UI tests for your Xcode project using XCTest.

Posts under XCTest tag

145 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Do I need to down load iOS 18 for testing on iPhone 11 having iOS version 17.6.1 ?
I am using two iPhone11 devices, having iOS v17.6.1 for automation testing using Seleniumv4.X-Appium v2.x. Today I updated the mac to Sequoia15 version which updated the XCode to v16. Just after this update, I could not able to run the previously running automation script. The console message is "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: Unable to launch WebDriverAgent. Original error: xcodebuild failed with code 70. This usually indicates an issue with the local Xcode setup or WebDriverAgent project configuration or the driver-to-platform version mismatch."
3
0
1k
Nov ’24
XCTest to test EndpointSecurity error,ES_NEW_CLIENT_RESULT_ERR_NOT_PRIVILEGED
I tried to use XCTest to test my own project that uses EndpointSecurity, but when I created the esClient I got an error:ES_NEW_CLIENT_RESULT_ERR_NOT_PRIVILEGED, indicating that it was not root. This makes it impossible for me to do coverage tests for the ESClient application. Is there any way I can implement this ESClient test? If so, how should I use it? The project is a macOS program, if I use gcov, but I find I can't get coverage. Using __gcov_flush will indicate that there is no symbol #if !TARGET_IPHONE_SIMULATOR NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES); NSString *documentsDirectory = [paths objectAtIndex:0]; setenv("GCOV_PREFIX", [documentsDirectory cStringUsingEncoding:NSUTF8StringEncoding], 1); setenv("GCOV_PREFIX_STRIP", "13", 1); #endif extern void __gcov_flush(void); __gcov_flush(); #endif
3
0
646
Sep ’24
[MT] DVTAssertions: ASSERTION FAILURE in IDEFrameworks/IDEFoundation/ProjectModel/ActionRecords/IDESchemeActionsInvocationRecord.m:458
I am creating a iOS library project using Xcode 15 in objective C lang which has dependency XCTest framework and Objective C run time and having UI test cases which can be use used in various projects. I am able to build and run unit test cases using Xcode fine but when try to run test using command line getting below issue Command env NSUnbufferedIO=YES xcodebuild clean test -workspace EnptyLibrary.xcworkspace -scheme EnptyLibrary -configuration Debug -destination 'platform=iOS Simulator,OS=latest,name=iPhone 15 Pro' -destination 'platform=iOS Simulator,OS=latest,name=iPad Pro (12.9-inch) (6th generation)' -testPlan EnptyLibrary -derivedDataPath build -disable-concurrent-destination-testing -retry-tests-on-failure=NO Error log ** TEST SUCCEEDED ** 2024-09-12 16:36:34.822 xcodebuild[8345:45388802] Writing error result bundle to /var/folders/12/rcxtcvjs08169cc_fzhyc8x40000gq/T/ResultBundle_2024-12-09_16-36-0034.xcresult 2024-09-12 16:36:34.832 xcodebuild[8345:45388802] [MT] DVTAssertions: ASSERTION FAILURE in IDEFrameworks/IDEFoundation/ProjectModel/ActionRecords/IDESchemeActionsInvocationRecord.m:458 Details: Assertion failed: !self.hasSaved Object: <IDESchemeActionsInvocationRecord: 0x112c21ce0> Method: -saveWithCompletionBlock: Thread: <_NSMainThread: 0x600000e441c0>{number = 1, name = main} Hints: Backtrace: 0 -[DVTAssertionHandler handleFailureInMethod:object:fileName:lineNumber:assertionSignature:messageFormat:arguments:] (in DVTFoundation) 1 _DVTAssertionHandler (in DVTFoundation) 2 _DVTAssertionFailureHandler (in DVTFoundation) 3 _DVTAssertionWarningHandler (in DVTFoundation) 4 -[IDESchemeActionsInvocationRecord saveWithCompletionBlock:] (in IDEFoundation) 5 -[IDESchemeActionsInvocationRecord saveSynchronouslyWithError:] (in IDEFoundation) 6 -[Xcode3CommandLineBuildTool saveResultBundleForError:triageWithSchemeTask:error:] (in Xcode3Core) 7 -[Xcode3CommandLineBuildTool _printError:schemeTask:andFailWithCode:] (in Xcode3Core) 8 -[Xcode3CommandLineBuildTool _printErrorWithSchemeTaskUserInfo:] (in Xcode3Core) 9 -[Xcode3CommandLineBuildTool _buildWithTimingSection:] (in Xcode3Core) 10 -[Xcode3CommandLineBuildTool run] (in Xcode3Core) 11 XcodeBuildMain (in libxcodebuildLoader.dylib) 12 -[XcodebuildPreIDEHandler loadXcode3ProjectSupportAndRunXcode3CommandLineBuildToolWithArguments:] (in xcodebuild) 13 -[XcodebuildPreIDEHandler runWithArguments:] (in xcodebuild) 14 main (in xcodebuild) 15 start (in dyld) zsh: abort env NSUnbufferedIO=YES xcodebuild clean test -workspace -scheme EnptyLibrary
0
0
589
Sep ’24
XCode 16 RC and Sequoia 15 RC - Running a XCTest UI test runs the test but does not open the simulator and initiating a record just hangs and never runs
When I run a XCTest test on an IOS Target device, the test runs but the simulator is not opened so I can follow the screens being displayed and the key strokes. And when I initiate a record for a test method, it just hangs and never runs or open a simulator. Is anyone else experiencing these same issues running XCTest? Is this legacy framework not working with XCode 16 by design?
3
1
819
Sep ’24
Unable to Launch Tests/UITests in Xcode 16 with iOS 18 Simulator
Hello, I'm unable to run any test cases in Xcode 16 with the iOS 18 simulator. Steps to Reproduce: Create a new iOS project in Xcode 16 using the default iOS app template with the testing option checked. Run any test or UI test case by clicking the "Play" icon in the Test navigator. Xcode gets stuck at the "Testing" state indefinitely. Environment: macOS 15 beta 8 Xcode 16 beta 6 iOS 18 beta 7 simulator
10
6
2.1k
Nov ’24
WeScan.ImageScannerControllerError iPhone Simulator
Hi everyone, I've got an issue with the iPhone simulator version 15 iOS 17.0. Every time a test attempts to take a photo, the app directs to the camera software and then this error appears. For my colleagues instead, they've been directed to the gallery. The only difference is that my machine is M1 while theirs is with Intel chip. I'm using MacOS 13.6.3, the same as them. Does anyone have any idea why this could happen? Thank you in advance.
0
0
340
Aug ’24
XCTest stuck and has error when take snapshot element on screen has much elements
Dear Apple Developer [Background] We are developing a service running on test mode based on XCTest Framework to support streaming, remote and automation testing with iOS devices. After we upgrade macOS Sonama and Xcode 15 version for upgrade XCTest for support iOS 17 around Jan-2024, we saw We noticed that our running on test mode services stuck and hanging frequently after a period of time. We have investigate and tried debug code and we found that main thread stuck and hanging when it call requesting snapshot of accessibility hierarchy for app. At the same time, we noticed that the application we were testing stuck and hanging too. [Environment] Application testing: Tiktok MacOS: Sonoma 14.5 Xcode: Version 15.4 (15F31d) Device test: iPhone 12 (iOS 14.8.1) Log detail: t = 1524.15s Wait for com.ss.iphone.ugc.Ame to idle t = 1536.54s Find the Application 'com.ss.iphone.ugc.Ame' t = 1536.54s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 1597.56s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1) t = 1597.56s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 1658.58s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2) t = 1658.58s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 1718.58s Collecting extra data to assist test failure triage t = 1718.58s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 1778.59s Requesting snapshot of accessibility hierarchy for app with pid 509 2024-08-19 14:16:44.990260+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0 2024-08-19 14:16:44.990428+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: XCTPerformOnMainRunLoop work timed out after 60.0s ((null)) (null) 0 0 t = 1843.71s Find the Application 'com.ss.iphone.ugc.Ame' t = 1843.71s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 1874.72s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1) t = 1874.72s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 1940.76s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2) t = 1940.77s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2005.80s Collecting extra data to assist test failure triage t = 2005.80s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2066.58s Requesting snapshot of accessibility hierarchy for app with pid 509 2024-08-19 14:21:33.019582+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0 2024-08-19 14:21:33.019687+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: Unable to perform work on main run loop, process main thread busy for 30.0s ((null)) (null) 0 0 2024-08-19 14:21:33.020647+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0 2024-08-19 14:21:33.020722+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: No element snapshot found for Application 'com.ss.iphone.ugc.Ame'[0.00, 0.00] -> (200.0, 259.0) ((null)) (null) 0 0 2024-08-19 14:21:33.020801+0700 WebDriverAgentRunner-Runner[489:97809] *** Assertion failure in -[XCPointerEventPath initForTouchAtPoint:offset:], XCSynthesizedEventRecord.m:263 t = 2291.08s Find the Application 'com.ss.iphone.ugc.Ame' t = 2291.08s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2352.10s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1) t = 2352.10s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2413.11s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2) t = 2413.11s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2473.12s Collecting extra data to assist test failure triage t = 2473.12s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2533.13s Requesting snapshot of accessibility hierarchy for app with pid 509 2024-08-19 14:29:19.552744+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0 2024-08-19 14:29:19.553054+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: XCTPerformOnMainRunLoop work timed out after 60.0s ((null)) (null) 0 0 t = 2598.27s Find the Application 'com.ss.iphone.ugc.Ame' t = 2598.28s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2659.29s Find the Application 'com.ss.iphone.ugc.Ame' (retry 1) t = 2659.29s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2720.31s Find the Application 'com.ss.iphone.ugc.Ame' (retry 2) t = 2720.32s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2780.33s Collecting extra data to assist test failure triage t = 2780.33s Requesting snapshot of accessibility hierarchy for app with pid 509 t = 2840.34s Requesting snapshot of accessibility hierarchy for app with pid 509 2024-08-19 14:34:26.806686+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0 2024-08-19 14:34:26.806828+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: Failed to get matching snapshot: XCTPerformOnMainRunLoop work timed out after 60.0s ((null)) (null) 0 0 2024-08-19 14:34:26.808114+0700 WebDriverAgentRunner-Runner[489:97809] Issue type: 0 2024-08-19 14:34:26.808205+0700 WebDriverAgentRunner-Runner[489:97809] Enqueue Failure: No element snapshot found for Application 'com.ss.iphone.ugc.Ame'[0.00, 0.00] -> (200.0, 259.0) ((null)) (null) 0 0 2024-08-19 14:34:26.809241+0700 WebDriverAgentRunner-Runner[489:97809] *** Assertion failure in -[XCPointerEventPath initForTouchAtPoint:offset:], XCSynthesizedEventRecord.m:263
1
0
671
Aug ’24
Deadlock in UIKit while injecting test bundle
Platform and Version iOS Development environment: Xcode 16.0 beta 5 (16A5221g), macOS 14.6.1 (23G93) Run-time configuration: iOS 18.0 beta 5 (22A5326g) Description of Problem Starting with iOS 18 SDK, test bundles containing a +load method that accesses UIScreen.mainScreen result in deadlock during test bundle injection. Also filed as FB14703057 I'm looking for clarity on whether this behavior is considered a bug in iOS or whether we will need to change the implementation of our app. Steps to Reproduce Create a new iOS app project using Objective-C and including unit tests Add the following code snippet to any .m file in the test target: @interface Foo: NSObject @end @implementation Foo + (void)load { UIScreen * const mainScreen = UIScreen.mainScreen; NSLog(@"%@", mainScreen); } @end Run the tests Expected Behavior As with iOS 17 & Xcode 15, the tests run to completion. Actual Behavior With iOS 18 & Xcode 16, deadlock during test bundle injection. stack_trace.txt
2
0
739
Sep ’24
How to get specific metric object from MetricKit using measure(metrics: metrics) results on XCUITest?
let metrics: [XCTMetric] = [XCTClockMetric(), // to measure time XCTCPUMetric(), // to measure cpu cycles XCTStorageMetric(), // to measure storage consuming XCTMemoryMetric(), ] let measureOptions = XCTMeasureOptions.default measureOptions.iterationCount = 1 measure(metrics: metrics) { //App flow } I want to get values of XCTCPUMetric, XCTMemoryMetric, XCTStorageMetric etc in any variable so that if want to send it further somewhere I can do it. Example - // let cpuMetric = CPU measure object should be here & I can get each information from this object. // let MemoryMetric = Memory measure object should be here & I can get each information from this object. But It's not available in XCUITest. We can only able to find it in the TestResult file. Please suggest any code available to get each metric object & value in the XCUITest rather than the test result.
0
0
458
Aug ’24
How specific metric object from measure(metrics: metrics) results on XCUITest using MetrixKit?
` let metrics: [XCTMetric] = [XCTClockMetric(), // to measure time XCTCPUMetric(), // to measure cpu cycles XCTStorageMetric(), // to measure storage consuming XCTMemoryMetric(), ] let measureOptions = XCTMeasureOptions.default measureOptions.iterationCount = 1 measure(metrics: metrics) { //App flow } ` I want to get values of XCTCPUMetric, XCTMemoryMetric, XCTStorageMetric etc in any variable so that if want to send it further somewhere I can do it. Example - // let cpuMetric = CPU measure object should be here & I can get each information from this object. // let MemoryMetric = Memory measure object should be here & I can get each information from this object. But It's not available in XCUITest. We can only able t find it in the TestResult file. Please suggest any code available to get each metric object & value in the XCUITest rather than the test result.
0
0
440
Aug ’24
XCode Cloud Aborting after 2 hours
Hi all, I have a UI Automation Test suite with a large number of test cases, that typically takes me 7 hours to run. When I setup the workflow on XCode cloud, I received the following error - "Build has exceeded the maximum allotted time. Each build can run for up to 120 minutes. If a build has not yet finished at this time, all incomplete actions will be cancelled." Is there any way to extend the build time? What is a build? I have two actions in my workflow. Build Action (that took 3 minutes) and the test step that took 2 hour 4 minutes. Am not clear how the 120 minute window is calculated.
1
0
550
Aug ’24
Forcing right to left text direction in Xcode UI test prevents sliders from moving
I'm using the following code to launch a UI test that forces a specific app language and moves a slider. I noticed that when forcing right to left text direction (for Arabic or Hebrew), during the UI test the slider doesn't move. The app content: struct ContentView: View { @State private var slider = 0.0 var body: some View { Slider(value: $slider, in: 0.0...1.0) } } The UI test: final class problemUITests: XCTestCase { func testExample() throws { let app = XCUIApplication() let locale = Locale(identifier: "ar") app.launchArguments += ["-AppleLanguages", "(ar)", "-AppleLocale", "ar"] if locale.language.characterDirection == .rightToLeft { app.launchArguments += ["-NSForceRightToLeftWritingDirection", "YES", "-AppleTextDirection", "YES"] } app.launch() app.sliders.element.adjust(toNormalizedSliderPosition: 0.3) } } Am I missing something or is there a workaround?
0
1
416
Jul ’24
Simulate hardware keyboard button press in Xcode UI test for iPad
XCUIElement has two methods named typeKey(_:modifierFlags:): the first one takes a String as an argument, the second one a XCUIKeyboardKey, which has constants like .downArrow. Now, even if the documentation says that both methods are "Available in macOS and in iPadOS 15 and later", when compiling the UI tests for iOS, the following line app.typeKey(.downArrow, modifierFlags: []) produces a compiler error Type 'String' has no member 'downArrow' Am I missing something or is there a workaround?
0
0
393
Jul ’24
UI Tests with mock server
I'm using Xcode Cloud for release builds and unit tests, and it works fine. Recently, I added some XCUITests, but to run it the way I need I started to use this mock server. Basically, the idea is you define before each test what responses you want to get for specific requests. It works like a charm locally. But if I want to run it in Xcode Cloud, it looks like the server has failed to launch. The mock server is a separately launched binary file. I think this is because this permission I get when launch it locally. So, my question is how to allow this kind of permission in Xcode Cloud to make my UITests works correctly? You can find a detailed instruction how server is launched on a github page I provided.
1
2
621
Aug ’24