Run Instruments through CI

Question

swiftBank OP

Created Nov ’19

Replies 1

Boosts 0

Views 1.5k

Participants 2

Hi,

Am trying to find a way to run Instruments like leaks, Core Animation (FPS) in CI to ensure that all PRs merge with the required levels for these metrics. In addition, I am looking to do some of these on device ie. FPS measure.

1. Are there command line options that can be used by CI to startup these instruments and run on Simulator and iOS Devices

2. are there ways to diff the output trace files to automate the checks on the metrics for eg FPS is atleast 60FPS

3. If anyone has done it, could you also talk to how it affects the CI performance for each PR?

Boost

Answer 1

Apple Designer OP

Apple

Jun ’20

While I'm not sure about running Instruments from the command line, with Xcode 12 you can write performance tests for animations and memory, which would let you have the same sort of performance checks you're interested in.

From the Xcode 12 Beta release notes:

Performance XCTests now support animation performance testing when using XCTOSSignpostMetric coupled with an animation os_signpost interval. The performance measurements returned include duration, three hitch-related metrics, and frame rate. To create an animation os_signpost interval, create a custom interval or use one of the provided UIKit intervals. The following shows an example performance test that measures the animation performance of scrolling an application. (55644042)

Code Block swiftfunc testScrollingAnimationPerformance() throws {
	let table = app.tables.firstMatch
	measure(metrics: [XCTOSSignpostMetric.scrollDecelerationMetric]) {
		table.swipeUp(velocity: .fast)
	}
}

We have a session in WWDC2020's testing collection that goes over how to use these new APIs. You can check it out here.

Also, here's a link to the performance testing documentation we have as well, which includes the list of metrics you can write performance tests around.

I hope this helps :)

1