Getting error measuring signpost metrics in XCUITests

Question

Created Mar ’20

Replies 5

Boosts 0

Views 1.5k

Participants 4

I'm running some XCUITests that utilize the measure(metrics:block:) call to gather signpost metrics (using XCTOSSignpostMetric) from the app being tested.

Running the tests one at a time works just fine, but when I try to run multiple tests consecutively, all tests after the first fail with the error: "Cannot activate instrument while another is already active"

Is there something that I should be doing after the measure block is run or in a tearDown() to clean up the metrics being measured? I couldn't find any documentation about cleaning up metrics, but it appears that the underlying instruments are not being cleaned up appropriately when a test case completes.

Thanks in advance for any help you can offer.

Boost

Answer 1

PeterSN OP

Apr ’20

Update: Here's an example of one of the errors we're seeing when we run multiple performance XCUITests consecutively:

*** Assertion failure in -[MXMOSSignpostMetric _constructProbe], /BuildRoot/Library/Caches/com.apple.xbs/Sources/MetricKit/MetricKit-60.60.1/MetricMeasurement/MetricMeasurement/metrics/MXMOSSignpostMetric.m:70

2020-04-02 10:51:51.779 PerformanceUITests-Runner[347:11285] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid parameter not satisfying: _startDate'

Again, the failure is occuring when using measure(metrics:block:) to gather signpost metrics.

Any help would be greatly appreciated! Thanks!

1

Answer 2

aman.toko OP

Jun ’20

I am facing a similar issue is there any update on this issue.

0

Answer 3

herzi OP

Jun ’20

I just ran into this issue and the only information on the web was this thread. Please go ahead and file a bug report at Apple: https://feedbackassistant.apple.com/

Here you can see how I reported this issue to be addressed: http://openradar.appspot.com/radar?id=4976981179367424

0

Answer 4

PeterSN OP

Jun ’20

Update: I have also opened a Developer Technical Support ticket for this issue. The latest update from the DTS Engineer is that the MetricMeasurement Engineering team is still investigating the issue, and that there's currently not a workaround. :(

I noticed some additional signpost metrics are being added in the upcoming Xcode/iOS, so maybe this issue will be fixed with those changes? I hope...

0

Answer 5

SoaurabhK OP

Aug ’20

Hi PeterSN, herzi, aman,

I think for now the solution is monkey patching it, as done here: https://github.com/SoaurabhK/XCMetrics

Cheers & Thanks,
Soaurabh

0