UITests with SwiftUI Failing to Find Off-Screen Elements in Xcode 14+

Question

JShroyer OP

Created Nov ’22

Replies 2

Boosts 1

Views 2.5k

Participants 3

Summary

I have filed feedback for this item, but I am curious whether this is an intended change or if there's a better way to address the behavior change - FB11764272.

UITests' XCTest behavior has changed with Xcode 14+ when looking for off-screen elements in a List component.

The Change

Below is how the behavior worked on the last two major versions of Xcode:

Xcode 13: Off-screen elements are visible to the UITest process and will scroll to those elements for interaction. e.g. calling tap() on an off-screen element.
- This is how UIKit TableViews work, even for cells that haven't been dequeued yet.
Xcode 14: Off-screen elements are NOT visible to the UITest process by default and will cause a test failure if an interaction is attempted. e.g. calling tap() on an off-screen element.
- This change is more in line with how Lazy Stacks worked in SwiftUI for Xcode 13.

The documentation for XCTUIElement's tap() method says the following: https://developer.apple.com/documentation/xctest/xcuielement/1618666-tap

If the element exists within a scrollable view but is offscreen, XCTest will attempt to scroll the element onscreen before performing the tap.

Also, while an old post (7 years ago), this Developer Forums Post thread #16810 has the following response from an Apple Frameworks Engineer: https://developer.apple.com/forums/thread/16810

You are not supposed to have to scroll manually. Interacting with an element not currently visible in the scroll view is expected to implicitly first scroll the element to be visible without requiring effort. If that's not working for you, please file a bug report (using the Report Bugs link at the bottom of this page). Thanks!

This was true for SwitUI Lists in Xcode 13 but isn't the case with Xcode 14+ (and it appears to continue to not work for Lazy Stacks within ScrollViews in SwiftUI).

Current Workaround

As my team's entire app is written in SwiftUI and we have an extensive UITest suite, we'll need to find every usage of an interaction API, like tap(), and wrap that with a custom method that first scrolls the entire length of the scroll view (at worst) waiting for the intended element to exist.

This involves a lot of work from a development perspective but also adds a lot of time to run through our extensive UITest suite for an already long-running suite.

Wrap Up

Was this change intentional? If so, what's the suggested way of scrolling to off-screen elements when UI testing a SwiftUI application or is the workaround described above the expected path forward?

Boost

Answer 1

Developer Tools Engineer OP

Apple

Nov ’22

Hello! Thanks for asking this.

XCTest's UI Testing system is built on top of Accessibility, which means the system is only aware of elements that have been loaded into the accessibility hierarchy at the time of each request. If an element is in the accessibility hierarchy but outside of the view, we use accessibility to scroll the view appropriately to bring the targeted element into frame.

It is possible that SwiftUI delays the loading of views into the accessibility hierarchy on long scrollviews to optimize for performance. This would prevent XCTest from being able to see that view until it has been loaded into the hierarchy.

You can see all of the elements that are in the current hierarchy by checking the debugDescription of your XCUIApplication: print(XCUIApplication().debugDescription)

This might be why you are having the experience you describe. If you would like us to dig further into this issue, feel free to file a report on Feedback Assistant with an .xcresult file attached showing your test failing to scroll to an off-screen element. We would be happy to look into this further and discuss with the SwiftUI team if there are any improvements that could be made here. https://feedbackassistant.apple.com/

0

Answer 2

raybeyer_adbsg OP

Jan ’23

I also see the behaviour that JShroyer describes: it's like a lazy loading mechanism. here it's just a list with 6 entries, so it is not only valid for long lists. The problem is: It's flaky! One time it works, one time not.

Using print(XCUIApplication().debugDescription) helps: because there I can see, that the off-screen element is not listed.

I now drilled it down: here it seems to be just the the iPhone 14 Simulator that doesn't work. And it's not just my local simulator, we have the same problem on the CI simulator. Using iPhone 13 (same size class and traits) works like a charm.

0