I'm trying to get a handle on XCTest performance measurement, but they don't seem to work the way I understand them (which is very possibly incorrect).
There are:
defaultMetrics: [XCTMetric]
- "Subclasses of XCTestCase
can override this property to change the default metrics."- which includes:
XCTClockMetric, XCTCPUMetric, XCTMemoryMetric, et al.
and:
defaultPerformanceMetrics: [XCTPerformanceMetric]
- "Subclasses of XCTestCase
can override this method to change the behavior of measure(_:)
."- which includes the sole:
wallClockTime: XCTPerformanceMetric
If I override:
override class var defaultMetrics: [XCTMetric] {
return [XCTMemoryMetric(), XCTCPUMetric(), XCTClockMetric()]
}
and call:
self.measure(work)
All I see in the report is Duration and Time.
Yet if I pass an array of metrics in,
self.measure(metrics: [XCTMemoryMetric(), XCTCPUMetric(), XCTClockMetric()], block: work)
it works and displays CPU Cycles and Memory Physical etc.
Shouldn't the measure() method use the defaults I've specified?
Also, if I've overridden defaultPerformaceMetrics(), those metrics are only displayed if I don't pass XCTMetrics to measure(metrics:).
Ultimately, what I want is a report that displays the XCTMetrics I specify as well as the XCTPerformanceMetrics. I know there are other perf metrics beyond just wallClockTime.