Custom XCTOSSignpostMetric not capturing performance data

Hello,

I'm attempting to automate some performance tests we currently do manually using signposts and Instruments.

It looks like XCTOSSignpostMetric is the perfect tool for the job, but I can't get it to play nicely. If I use the pre-defined signpost metric constants (customNavigationTransitionMetric, scrollDecelerationMetric, etc), it works fine. If I use a custom signpost using the

XCTOSSignpostMetric.init(subsystem: category: name:)

initializer, nothing happens.

The documentation is very sparse on this topic and Googling, Binging, Githubing and Twittering have come up empty.

I reduced the issue to the smallest example I could ( https://github.com/tspike/SignpostTest ).

What am I doing wrong?

Thanks, Tres

Environment details:

  • macOS 11.6
  • Xcode 12.5.1
  • iOS 14.6
  • iPhone SE 1st Gen

In the app:

...
let signpostLog = OSLog(subsystem: "com.tspike.signpost", category: "signpost")
let signpostName: StaticString = "SignpostTest"

@main
struct SignpostTestApp: App {
	init() {
		os_signpost(.begin, log: signpostLog, name: signpostName)
		DispatchQueue.main.asyncAfter(deadline: .now() + .milliseconds(500), execute: {
			os_signpost(.end, log: signpostLog, name: signpostName)
		})
	}
	...
}

In the test

   func testSignposts() throws {
    let app = XCUIApplication()
//		No performance data
		let metric = XCTOSSignpostMetric.init(subsystem: "com.tspike.signpost",
											 category: "signpost",
											 name: "SignpostTest")
		
//		Works as expected
//		let metric = XCTOSSignpostMetric.applicationLaunch

		let options = XCTMeasureOptions()	
		options.iterationCount = 1
		measure(metrics: [metric], options: options, block: {
			app.launch()
		})
	}

Hello, @tresebay! Did you try adding some delay when running performance tests?

_ = app.wait(for: .notRunning, timeout: 3)

This can stabilise metrics collection.

Custom XCTOSSignpostMetric not capturing performance data
 
 
Q