XCUITest

Has anyone had any luck using the new UI testing yet? I would love to port all of the asynchronous javascript automation testing since in theory this will work better, but I haven't been able to get very far before tests fail with

t = 8.50s Wait for app to idle

t = 38.52s Assertion Failure: UI Testing Failure - App failed to quiesce within 30.0s

/Users/jos/GIT/***/***/***/***/xxxUITests.swift:41: error: -[xxxUITests.xxxUITests testDropbox] : UI Testing Failure - App failed to quiesce within 30.0s


Has ANYONE had any luck using this? This is even more frustrating than automating with javascript!


jos

Replies

Can anyone from Apple comment on the conditions that cause this to happen? I have been able to get further by removing code that updates the UI in the application that I am testing. Obviously this isn't a solution as I do need to update my GUI but it does point to what is happening. I can confirm that the main thread returns after updating in a reasonable amount of time (well within the 30 seconds, probably within the range of 1 second) yet the testing script hangs failing to realize that it can continue.


The case that I am testing here is logging on to Dropbox. After I sign in, my application retrieves the file information in my account (on a background thread) and then updates the GUI (on the main thread) to reflect this. I can get the test to proceed if my application does not update the GUI. My application is very responsive and nothing seems wrong with it. I just can't use the testing framework.


Any thoughts or input into how to proceed?


Thanks.

If the application performs enough work on the main thread continuously to not consider itself idle, I would be curoius to know how that affect power / batterly life? You mention that this is likely to do with status updates from background operations being handled by the main thread. I would also be curious to know if the frequency of updates is higher than it would have to be.


That said, if this is an issue for you, I would advice you to file a bug report.


http://developer.apple.com/bugreporter/


Thanks!

I just replied in another message what happened. I kept debugging this and actually traced it down to a point where I was calling endRefreshing on a UIRefreshControl without a preceding startRefreshing. Once I checked the refreshing attribute, the test was able to succeed. While this isn't documented anywhere and my application seemed to run just fine in this case, the test clearly couldn't handle that.

And thank you joar for responding!

Would the Time Profiler give insight as to why the main thread isn't being seen as "idle?" What triggers would prevent the test runner from seeing an app as "idle?"


Instrumenting my app, which is having a similar time out behavior, the main thread is quiet after the view controller draws its table view.

How on earth did you track this down? What were you seeing in the debugger that even gave you this breadcrumb to follow! :-)

Hopefully joar can give insight because my main thread was quiet as well. I also put debugging log statements in showing that execution left the main thread quickly. It was just trial and error until I found out that it was the refresh control that caused the test simulator to "think" the main thread was busy.

And in response to your previous question, I just commented out everything that affected the main thread and slowly put it back in until I could limit it down to the single line. Tedious but luckily effective!

Thanks this is very helpful.


I've followed your lead and commented out the view controller life cycle methods (viewDidLoad, viewWillAppear:, viewDid...) on the view controller under tests. My tests have started "working," but of course there's bugs left and right w.r.t. other functionality. It seems like this is solid troubleshooting methodology.

UIRefreshControl bit me, too.


I had to put guard clauses around my async callback :

if (self.refreshControl.isRefreshing) {
   [self.refreshControl endRefreshing];
}

That's what I did as well. Surely that's a bug with the UIRefreshControl because my app worked just fine despite that. I'm glad that I could save you some time! I couldn't understand how people had been able to use the testing framework all this time which is why I finially started to debug the code on my end. Hopefully it's the only control that has this strange effect on testing.

If you can't figure out issues of this nature, please file bug reports.

radr://22158864


Joar,

The bug report points to this discussion thread. I'm sure the screener will let me know they need sample project. Question: Is there any point in having jos dupe the bug? I've heard dev tools doesn't like dupes.

Thanks for your bug report!


Yes, the screener (ie. someone on my team) will reach out to you for additional informaiton if required. We love sample projects, or concrete steps for reproducing reported problems (obviously).


We don't mind duplicate bug reports, as long as they're not simply a "me too". If you have any reason to believe that you could contribute different / additional information to help understanding some issue (and that's very often possible), then by all means file another bug report. Even a "me too" report helps us gauge how many developers are affected by any given issue, so I'm not particularly opposed to those either. File more bug reports! ;-)

I have a `UIView` animation with animation option `Repeat`. It is causing the app to never be "quiesce". This animation isn't really a big battery drain, and CPU level seems fine.


Any idea if this is a bug, or such animations must be disabled during UI Testing? (since it seems UI Tests work well with navigation animation)


let radarCircle = CircleView(frame: bounds)
addSubview(radarCircle)
let baseCircle = CircleView(frame: bounds)
addSubview(baseCircle)
UIView.animateWithDuration(1.5, delay: 0.0, options: .Repeat, animations: {
    radarCircle.transform = CGAffineTransformMakeScale(2, 2)
    radarCircle.alpha = 0
}, completion: nil)


Thanks,

Maz