Missing code coverage with Xcode 10

We are switching to Xcode 10 and are noticing some issues with Xcode 10's code coverage. Entire swaths of code are getting no coverage. So far I have noticed


* Computed vars

* class functions in extensions

* guard statements in extensions


For example:


import UIKit

extension Date {
  func dayNumberOfWeek() -> Int? {
  return Calendar.current.dateComponents([.weekday], from: self).weekday
  }
}

extension UIView {

  // Testing with this function compiled will cause no code coverage
  // for steveWasHereHelper to be generated.
  //
  // However, if I comment out steveWasHere, then code coverage for
  // steveWasHereHelper is generated, despite that this function is also
  // called by wyattWasHere
  @discardableResult public class func steveWasHere(value: Int) -> Bool {
  return self.steveWasHereHelper(value: value)
  }

  // The usage of Date here is to try and rule out compiler optimization
  @discardableResult public class func steveWasHereHelper(value: Int) -> Bool {
  let d = Date().dayNumberOfWeek()

  let testValue = value + d!
  if testValue >= 42 {
  return true
  }

  return false
  }

  fileprivate class func neverCalled() {
  print("This is never called")
  }

  // steveWasHereHelper is also called here
  @discardableResult public class func wyattWasHere(value: Int) -> Bool {
  return self.steveWasHereHelper(value: value)
  }
}


In this example, steveWasHereHelper will not get any coverage. It won't be green, it won't be red, its as if the code is not called. If I remove all tests EXCEPT for the direct test of steveWasHereHelper, it still gets no coverage.


HOWEVER, if I then comment out steveWasHere, THEN steveWasHereHelper shows coverage.


I have opened a Radar with Apple on this issue (45468318) but I was wondering if anyone else has seen this (This is not the only example of issues with code coverage in Xcode 10) and if there is a workaround.


I have also tested this in 10.1b3, getting the same results.


self. was added to the function calls just to see if it had any impact.


Narrator: It didn't

Replies

Hi Steve,

I am seeing a similar issue on Xcode 10.0 (10A255). Getting no coverage for an extension of a struct with a where clause.

I don't have a workaround yet - will post if I find one.

I am seeing similar issues. Actually not only that, what I am noticing :

1- Coverage number varies between multiple runs on XC 10 on the same binary. Like in first run it shows x% vs in another run it will show y% keeping the same code.

2- Coverage number/Number of tests varies while running on 11.4 simulator and 12.0 simulator, both ran on XC 10

3- Number of tests also a little different like in some of my run it was 5507 tests vs in some runs it was 5506.


XC 10 certainly came with lots of bugs.


Please provide more input if you faced similar issues or found any workaround or solutions @stevebay hybridcatttstevebay

I'm finding code coverage with Xcode 10 very frustrating. I use it to help find areas that have no test coverage, but it's been very unreliable.


I have restarted Xcode, deleted derived data, turned code coverage off/on, changed what was covered (all vs specific targets) and nothing produces deterministic results.


In the current state of my project, Xcode reports large blocks of code as never being hit (right-hand gutter in source files show code counts in only parts of the file). Yet coverage reports show the file as being covering 100%. Setting breakpoints in numerous areas in that file are never hit, so the 100% claim is a lie.


Sometimes, the values as reported in the right-hand gutter seems correct. But then the reported coverage is less than 100%.


While massively painful, I think I will have to resort to literally adding breakpoints to every individual code path and prove I have ample test coverage.

It's good to know it is not just us. I have opened a second TSI with Apple on this, also see:


https://bugs.swift.org/browse/SR-9145?jql=text%20~%20%22code%20coverage%22