XCode 9 XCUITests suddenly fail

When we installed XCode 9 and went on validating our installation, we noticed some of our UI tests suddenly failed so we investigated what could be wrong but it makes no sense. For example:

UI Test Activity:
Assertion Failure: Helper.m:320: Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate `exists == 1` for object StaticText".


The Helper function is the following:

- (void)exists:(id)object {
    NSPredicate *exists = [NSPredicate predicateWithFormat:@"exists == 1"];
    [self expectationForPredicate:exists evaluatedWithObject:object handler:nil];
    [self waitForExpectationsWithTimeout:PREDICATE_WAIT_TIMEOUT handler:nil];
}

Called from here:

- (void)assertErrorVisible:(BOOL)visible {
    NSPredicate *identifierPredicate = [NSPredicate predicateWithFormat:@"identifier == 'label_printer_feedback'"];
    NSPredicate *lengthPredicate = [NSPredicate predicateWithFormat:@"label.length > 0"];
    [self.helper exists:[self.app.staticTexts elementMatchingPredicate:[NSCompoundPredicate andPredicateWithSubpredicates:@[identifierPredicate, lengthPredicate]]]];
}

We decided to debug and check the contents of object with 'po' and it does find a match:

Find: Elements matching predicate 'identifier == "label_printer_feedback" AND label.length > 0'
      Output: {
        StaticText, 0x7fcc5cca3f60, traits: 8589934656, {{311.0, 299.0}, {402.0, 20.0}}, identifier: 'label_printer_feedback', label: 'Kan geen verbinding met de printer maken.'
      }


We used to have a lot of these problems 2 years ago with XCode 7. Regression in XCode 9?

Replies

From the data that you show here, this does look like a possible bug in XCTest. Are you able to reproduce this problem in a sample project that you could attach to a Radar?


Note also that XCTest in Xcode 9 has a new `-waitForExistenceWithTimeout:` method that you could adopt in place of your current `-exists:` method:


https://developer.apple.com/documentation/xctest/xcuielement/2879412-waitforexistencewithtimeout?language=objc