Hello Developers,
I regularly use Xcode Cloud to run unit tests, but since the release of Xcode 16, the testing process has become significantly slower.
I am seeking advice on how to improve the situation.
[ Details ]
With Xcode 15.4, unit tests took about 1 hour to complete.
However, since upgrading to Xcode 16, the tests have not finished even after 2 hours.
The blog post linked below also mentions significant overhead in running unit tests on Xcode Cloud, even with Xcode 15.4:
https://dwango.github.io/articles/2024-07_nicoiphone_xcode_cloud/#%E3%83%86%E3%82%B9%E3%83%88%E5%AE%9F%E8%A1%8C%E6%99%82%E9%96%93%E3%81%8C%E5%A2%97%E5%8A%A0%E3%81%97%E3%81%A6%E3%81%97%E3%81%BE%E3%81%86%E5%95%8F%E9%A1%8C
According to the blog, while reviewing the xcodebuild-test-without-building.log, it was observed that the time from start to finish was about 4 minutes.
However, the overall “Run xcodebuild test-without-building” process took 23 minutes, with around 20 minutes of apparent idle time.
It seems that the idle time has increased even further with Xcode 16, resulting in more than twice the testing time.
[ Question/Request ]
Is there any way to reduce the overhead when running unit tests on Xcode Cloud, especially with the increased overhead in Xcode 16?
Any advice or solutions for improving the performance would be greatly appreciated.
Best regards,
Post
Replies
Boosts
Views
Activity
Hi,
Our team uses Xcode Cloud to run unit tests, and since around November, we’ve been frequently experiencing timeouts with the following type of process:
@Test func hogeTest() async {
// do something
hoge.do()
// wait until done
await wait(condition: { hoge.isDone })
// test result
#expect(hoge.isSucceeded)
}
private func wait(condition: () async -> Bool, timeout: TimeInterval = 0.5, pollingInterval: TimeInterval = 0.01) async throws {
let deadline = Date().addingTimeInterval(timeout)
while Date() < deadline {
if await condition() { return }
try await Task.sleep(nanoseconds: UInt64(round(pollingInterval * TimeInterval(NSEC_PER_SEC))))
}
Issue.record("timeout")
}
Although sleep is supposed to wait for only a few milliseconds, there are cases where it takes more than 10 seconds, leading to a timeout.
What could be causing this instability in the sleep duration? Additionally, if there are other recommended ways to implement polling and waiting in Swift Testing, I would appreciate it if you could share them.
A feedback report (FB15899163) has already been submitted.
Best regards,