UI Tests on real device broken on iOS 15.4

Question

UI Tests seem to be broken when run on an iPhone running iOS 15.4.

Steps to reproduce:

Create a fresh app project in Xcode 13.3 including UI tests
Select an attached iPhone running iOS 15.4
Run the UI test testExample which only starts the app, nothing more.
Restart the attached phone
Try to run testExample again.

Expectation:

Test succeeds again

Actual:

When launching the app, XCTest is internally first trying to terminate the app and gets stuck. After 60s the tests fail.

Logging (app id and home folder are removed for privacy reasons):

TESTUITests-Runner[341:7025] Running tests...
Test Suite 'Selected tests' started at 2022-03-17 14:53:36.855
Test Suite 'TESTUITests.xctest' started at 2022-03-17 14:53:36.855
Test Suite 'TESTUITests' started at 2022-03-17 14:53:36.856
Test Case '-[TESTUITests.TESTUITests testExample]' started.
    t =     0.00s Start Test at 2022-03-17 14:53:36.856
    t =     0.07s Set Up
    t =     0.07s Open <app_id>.TEST
    t =     0.11s     Launch <app_id>.TEST
    t =     0.11s         Terminate <app_id>.TEST:269
<UserHome>/Workspace/TEST/TESTUITests/TESTUITests.swift:28: error: -[TESTUITests.TESTUITests testExample] : Failed to terminate <app_id>.TEST:269: Failed to terminate <app_id>.TEST:0
    t =    60.29s Tear Down
Test Case '-[TESTUITests.TESTUITests testExample]' failed (60.294 seconds).
Test Suite 'TESTUITests' failed at 2022-03-17 14:54:37.150.
	 Executed 1 test, with 1 failure (0 unexpected) in 60.294 (60.294) seconds
Test Suite 'TESTUITests.xctest' failed at 2022-03-17 14:54:37.150.
	 Executed 1 test, with 1 failure (0 unexpected) in 60.294 (60.295) seconds
Test Suite 'Selected tests' failed at 2022-03-17 14:54:37.151.
	 Executed 1 test, with 1 failure (0 unexpected) in 60.294 (60.296) seconds

If I manually start the app once, then the UI tests will run again afterwards for a while (including restarting the app). At some point they get stuck again and we are not sure why, yet. This makes it really hard to run tests in our CI because we can't keep sending someone into the server room to manually start the app...

Has anybody else encountered this and found a solution? It works fine on iOS 15.1.1.

3.2k

Posted by

Infinite

Reply

Same issue here on iOS 15.4 (19E241) with a real device (iPhone 12).

Works fine in an iOS 15.4 simulator in Xcode 13.3 (13E113).

Also worked fine on that iPhone 12 on all previous iOS 15 releases (15.3.1, 15.3, 15.2.1, 15.2, 15.1.1, 15.1, 15.0.2, 15.0.1, 15.0).

—
phirk
Still an issue in iOS 15.4.1 (19E258).

—
phirk
I'm seeing this issue as well, and unfortunately the activate followed by launch approach has other issues with the environment variables specified in a test not applying to the launch. Has anyone seen if this still happens on 15.5?

—
noahmartin

Add a Comment

Answer 1

Use app.activate() before calling app.launch() on 15.4+

Posted by

PlessioTihsrah

@PlessioTihsrah We tried app.activate() and app.launch() in isolation, but not the combination...

Your suggestion works for us; our UI tests are green again on iOS 15.4.1 now. Thank you!

—
phirk

Add a Comment

Answer 2

Was there a fix for this issue in iOS 16? Not able to see this myself in the version 16.

Posted by

anandga11

Add a Comment

Answer 3

It seems like this issue has been resolved in iOS 16 and Xcode 14, but please let us know if this issue continues by filing a bug in Feedback Assistant.

Posted by

Developer Tools Engineer

I am still having this issue in the Xcode Version 15.1 (15C65) and whole tests passed but at the end I am getting this error Failed to terminate com.XXXXXXXXXXXX: Failed to terminate com.XXXXXXXXXXXX

—
saimahanif

Add a Comment

UI Tests on real device broken on iOS 15.4

Replies