How to write a unit test which passes if a function throws?

I want to define a function which returns a value if successful (typically, if the arguments are valid) and throws if it encounters a problem (such as the arguments being invalid). I want to write a unit test to verify that invalid arguments cause errors.


There doesn't seem to be an XCTAssertThrows for Swift tests.


I've tried written a unit test in this form:

func testInvalidArgumentsResultInErrors()
{
 // set up system under test and invalid arguments
  do
  {
    let _ = try systemUnderTest.function(invalidArguments)
    XCTAssert(false)
  }
  catch
  {
    XCTAssert(true)
  }
}

It works, but doesn't look pretty, and if I want to test for a specific error type that means another catch closure. Is there a more elegant way of writing this?


In the hope of at least making each test look a bit better, I tried writing the above as another function, so I can do something like this:

XCTAssert(tryFunctionExpectingFailure(systemUnderTest.function(invalidArguments)))


func tryFunctionExpectingFailure<T>(myFunction:( () -> T )) -> Bool
{
 do
  {
   let _ = try myFunction()
   return false
  }
 catch
 {
   return true
 }
}

This doesn't work because the function doesn't believe that myFunction throws. How do I pass a function in such a way that the compiler knows it might throw?

Accepted Reply

Here are some XCTest style assertions I wrote for checking for thrown Swift errors. Hopefully by the time Xcode 7 is final, there will be built-in versions of these.


//
//  XCTAssertSwiftError.swift
//
//  Created by LCS on 6/17/15.
//  Released into public domain; use at your own risk.
//

import XCTest


func XCTempAssertThrowsError(message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do
    {
        try block()
      
        let msg = (message == "") ? "Tested block did not throw error as expected." : message
        XCTFail(msg, file: file, line: line)
    }
    catch {}
}


func XCTempAssertThrowsSpecificError(kind: ErrorType, _ message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do
    {
        try block()
      
        let msg = (message == "") ? "Tested block did not throw expected \(kind) error." : message
        XCTFail(msg, file: file, line: line)
    }
    catch let error as NSError
    {
        let expected = kind as NSError
        if ((error.domain != expected.domain) || (error.code != expected.code))
        {
            let msg = (message == "") ? "Tested block threw \(error), not expected \(kind) error." : message
            XCTFail(msg, file: file, line: line)
        }
    }
}


func XCTempAssertNoThrowError(message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do {try block()}
    catch
    {
        let msg = (message == "") ? "Tested block threw unexpected error." : message
        XCTFail(msg, file: file, line: line)
    }
}


func XCTempAssertNoThrowSpecificError(kind: ErrorType, _ message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do {try block()}
    catch let error as NSError
    {
        let unwanted = kind as NSError
        if ((error.domain == unwanted.domain) && (error.code == unwanted.code))
        {
            let msg = (message == "") ? "Tested block threw unexpected \(kind) error." : message
            XCTFail(msg, file: file, line: line)
        }
    }
}


And here are samples of calling them with a trailing closure:

import XCTest

enum SampleError: ErrorType {case ItCouldBeWorse, ThisIsBad, ThisIsReallyBad}

func sampleThrowingFunc(v: Int) throws -> Int
{
    if (v == 0) {throw SampleError.ItCouldBeWorse}
    if (v < 0) {throw SampleError.ThisIsBad}
    if (v > 255) {throw SampleError.ThisIsReallyBad}
    return v
}


class SwiftErrorTestingSample: XCTestCase
{
    func test_Examples()
    {
        let a = -35
       
        XCTempAssertThrowsError() {try sampleThrowingFunc(a)}
       
        XCTempAssertThrowsSpecificError(SampleError.ThisIsBad) {try sampleThrowingFunc(a)}
       
        XCTempAssertNoThrowError("customized failure message") {
           
            let x = Int(arc4random() % 256)
            let y = try sampleThrowingFunc(x)
       
            XCTAssert(x == y, "")
        }
       
        XCTempAssertNoThrowSpecificError(SampleError.ThisIsReallyBad) {try sampleThrowingFunc(a)}
    }

}


For the versions which match to a "specific" error, the provided ErrorType instance is converted to an NSError and the domain and code are compared to any errors which result from calling the closure. So it will match the enum type and specific member/case of a Swift ErrorType enum, but for enums where the members also have associated values attached it won't try to match those associated values.

Replies

Answer to the second question turns out to be a function declaration of:

func tryFunctionExpectingFailure<T>(myFunction:( () throws -> T )) -> Bool

However, in order to make this work if myFunction has arguments, myFunction now has to be a curried function.


Alse I don't think I can pass an expected error type into this tryFunctionExpectingFailure function as a second argument, so the function returns true only if the thrown error is of the expected type. I think I'm stuck with writing a do-try-catch in every test.

Here are some XCTest style assertions I wrote for checking for thrown Swift errors. Hopefully by the time Xcode 7 is final, there will be built-in versions of these.


//
//  XCTAssertSwiftError.swift
//
//  Created by LCS on 6/17/15.
//  Released into public domain; use at your own risk.
//

import XCTest


func XCTempAssertThrowsError(message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do
    {
        try block()
      
        let msg = (message == "") ? "Tested block did not throw error as expected." : message
        XCTFail(msg, file: file, line: line)
    }
    catch {}
}


func XCTempAssertThrowsSpecificError(kind: ErrorType, _ message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do
    {
        try block()
      
        let msg = (message == "") ? "Tested block did not throw expected \(kind) error." : message
        XCTFail(msg, file: file, line: line)
    }
    catch let error as NSError
    {
        let expected = kind as NSError
        if ((error.domain != expected.domain) || (error.code != expected.code))
        {
            let msg = (message == "") ? "Tested block threw \(error), not expected \(kind) error." : message
            XCTFail(msg, file: file, line: line)
        }
    }
}


func XCTempAssertNoThrowError(message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do {try block()}
    catch
    {
        let msg = (message == "") ? "Tested block threw unexpected error." : message
        XCTFail(msg, file: file, line: line)
    }
}


func XCTempAssertNoThrowSpecificError(kind: ErrorType, _ message: String = "", file: String = __FILE__, line: UInt = __LINE__, _ block: () throws -> ())
{
    do {try block()}
    catch let error as NSError
    {
        let unwanted = kind as NSError
        if ((error.domain == unwanted.domain) && (error.code == unwanted.code))
        {
            let msg = (message == "") ? "Tested block threw unexpected \(kind) error." : message
            XCTFail(msg, file: file, line: line)
        }
    }
}


And here are samples of calling them with a trailing closure:

import XCTest

enum SampleError: ErrorType {case ItCouldBeWorse, ThisIsBad, ThisIsReallyBad}

func sampleThrowingFunc(v: Int) throws -> Int
{
    if (v == 0) {throw SampleError.ItCouldBeWorse}
    if (v < 0) {throw SampleError.ThisIsBad}
    if (v > 255) {throw SampleError.ThisIsReallyBad}
    return v
}


class SwiftErrorTestingSample: XCTestCase
{
    func test_Examples()
    {
        let a = -35
       
        XCTempAssertThrowsError() {try sampleThrowingFunc(a)}
       
        XCTempAssertThrowsSpecificError(SampleError.ThisIsBad) {try sampleThrowingFunc(a)}
       
        XCTempAssertNoThrowError("customized failure message") {
           
            let x = Int(arc4random() % 256)
            let y = try sampleThrowingFunc(x)
       
            XCTAssert(x == y, "")
        }
       
        XCTempAssertNoThrowSpecificError(SampleError.ThisIsReallyBad) {try sampleThrowingFunc(a)}
    }

}


For the versions which match to a "specific" error, the provided ErrorType instance is converted to an NSError and the domain and code are compared to any errors which result from calling the closure. So it will match the enum type and specific member/case of a Swift ErrorType enum, but for enums where the members also have associated values attached it won't try to match those associated values.

Excellent stuff. Thanks for these - I hope Swift gets an official version of these soon too, but this is good.