In an XCTest, setting continueAfterFailure to true in setUp should cause the test to stop executing as soon a failed assertion occurs. This works correctly unless the test is marked async.
In that case, what I've seen is that execution of the failed test continues, and another test starts to execute in parallel. You can see this by pausing execution and seeing two cooperative threads actively executing two different tests. This can cause weird problems due to XCTestCase classes often not being reentrant. It seems that tests are executed concurrently only if they are async, one of them fails and continueAfterFailure is set to false.
It almost seems like the "host" thread for the test is properly stopped, but the cooperative thread that runs the async test continues on, while the "host" thread moves on and starts the next async test, which results in two cooperative threads running tests in parallel.