Swift, XPC, and... segmentation faults?

I thought Swift wasn't supposed to get them, which is part of the reason why I chose to use it for my network extension. But we're getting crashes occasionally, that look like:

Thread 4 Crashed::  Dispatch queue: com.apple.NSXPCConnection.user.endpoint

0   com.kithrup.MyApp.NExt                  0x102c4ffe2 MyExt.sendData(_:data:completion:) + 610
1   com.kithrup.MyApp.NExt                  0x102c5091f @objc MyExt.sendData(_:data:completion:) + 255
2   Foundation                              0x7ff81ef97490 __NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S3__ + 10
3   Foundation                              0x7ff81ef3fa1f -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] + 2322
4   Foundation                              0x7ff81eef641e message_handler + 206
5   libxpc.dylib                            0x7ff81de24b6c _xpc_connection_call_event_handler + 56
6   libxpc.dylib                            0x7ff81de23947 _xpc_connection_mach_event + 1382
7   libdispatch.dylib                       0x7ff81df2e3b1 _dispatch_client_callout4 + 9
8   libdispatch.dylib                       0x7ff81df47041 _dispatch_mach_msg_invoke + 445
9   libdispatch.dylib                       0x7ff81df341cd _dispatch_lane_serial_drain + 342
10  libdispatch.dylib                       0x7ff81df47b77 _dispatch_mach_invoke + 484
11  libdispatch.dylib                       0x7ff81df341cd _dispatch_lane_serial_drain + 342
12  libdispatch.dylib                       0x7ff81df34e30 _dispatch_lane_invoke + 417
13  libdispatch.dylib                       0x7ff81df3eeee _dispatch_workloop_worker_thread + 753
14  libsystem_pthread.dylib                 0x7ff81e0e1fd0 _pthread_wqthread + 326

The XPC method is func sendData(_: UUID, data: Data?, completion: @escaping (_: Error?) -> Void)

It's crashing on address 0x10, so pretty clearly a NULL-dereference.

Since this is happening in my extension, it's in Swift (as I said above), so I have no idea what could be NULL without the compiler yelling at me first.

Swift helps avoid memory management bugs but it can’t completely prevent them. There are two big holes in the Swift’s memory management defences:

  • Unsafe constructs, including any calls to a C-based language, which means most of the OS

  • Concurrency

XPC code can is subject to both (-:

As to what’s happening in your specific case, it’s hard to say without more info. To start, please post a full crash report. See Posting a Crash Report for advice on how to do that.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

None of the arguments is ever modified by something else; the most that happens is that references are added and removed. sigh.

Here's the raw-ish IPS. I'm sure concurrency is the problem.

Oh, and the textified version.

I'm sure concurrency is the problem.

In that case, have you tried TSan?

Looking at your crash report, I see this:

Thread 4 Crashed::  Dispatch queue: com.apple.NSXPCConnection.user.endpoint
0   com.kithrup.MyExt.NExt … NExt.handleData(_:data:completion:) + 610

+ 610 is a long way in to your method. If you symbolicate the log, what line does that map to? And what’s the code on that line?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I tried thread sanitizer, unfortunately it fails too much on the 3rd party C++ code the project uses.

The Swift method in question here has a closer which itself has a closure.

Also, is there a better way of getting the line number from the address than using lldb or objdump to disassemble with lines? ;) It's hit and miss, unfortunately.

I tried thread sanitizer, unfortunately it fails too much on the 3rd party C++ code the project uses.

Hmmm, that’s not a good sign. If you temporarily stub out that library, do you still see these crashes?

there a better way of getting the line number from the address

Most of the time that happens automatically, presuming you have your .dSYM files set up correctly. For all the gory details, see Adding Identifiable Symbol Names to a Crash Report.

Oh, one other hint: For testing purposes you can implement a loopback XPC listener, allowing you to stress test your XPC subsystem in a sensible environment, like an XCTest. For info on how to do that, see TN3113 Testing and debugging XPC code with an anonymous listener.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

We use cmake, so it was a bit more difficult -- and CircleCI, so I had to make sure the dsym files got archived. Did that. Re-confirmed it's this code:

        guard let flowData = self.flowSets[id] else {
            os_log(.error, log: Self.log, "Could not find flow %{public}@", id.uuidString);
            completion(RedirectorError.flowNotFound)
            return
	}

id is a UUID; self.flowSets is private var flowSets : [UUID : FlowData] = [:] (where FlowData is a simple class wrapping some flow-related information). This particular method is invoked via XPC from a daemon, with a signature of func handleTCPData(_ id: UUID, data: Data?, completion: @escaping (_: Error?) -> Void). The daemon code is in ObjC++, and it checks to ensure that the UUID is non-nil.

So I have no idea why it is segfaulting!

Which specific line does it point to?

One trick here is to break the statements up into multiple lines to help isolate it. For example:

let flowDataQ = self.flowSets[id]
guard flowData = flowDataQ else {
    …

A completely different approach is to enable core dumps on your daemon so that, when it crashes, you have something to debug.

Also, can you post a full crash report here. See Posting a Crash Report for advice on how to do that.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

The specific line is the guard statement, so yeah, I was going to try to break it up. However, I disassembled at one point:

         * At +521, %r13 is 0x0.                                                                                              
         * Redirector.o[0x8c70] <+512>:  callq  0x8c75                    ; <+517> [inlined] Redirector.flowSets.getter : Swift.Dictionary<Foundation.UUID, FlowData> + 27 at Redirector.swift:651:43                                                                                                 
         * Redirector.o[0x8c75] <+517>:  movq   (%r14,%rbx), %r13                                                             
         * Redirector.o[0x8c79] <+521>:  cmpq   $0x0, 0x10(%r13)                                                              
         * Redirector.o[0x8c7e] <+526>:  je     0x8d35                    ; <+709> at Redirector.swift:651:43

So I assume that %r13 has self.flowSets[id] (rather than id). For some reason, however, I didn't disassemble enough to have %r14 and %rbx annotated.

Swift, XPC, and... segmentation faults?
 
 
Q