Significant performance drop switching from Kernel to System Extension

We have a System Extension that fetches packets for various VPN protocols that our app supports and then hands them off to various XPC services (started and maintained by our app) that implement the actual protocols (e.g. IPSec, OpenVPN, SSL…). This design allows us to easily use the actual protocol implementation with versions that don't use a System Extension.

On older macOS releases, instead of a System Extension, we used a KEXT and root process to fetch packets, but also handed these off to the same XPC services.

The issue
With the new System Extension → XPC design, we're seeing a significant throughput performance hit once network speeds exceed a certain threshold, which we need to address.

Questions
Our old design would drop outgoing packets if they weren't being processed fast enough (as we must grab them from the kernel either way). It's unclear what the SysExt does in this case - are they dropped, is there a buffer, could this be cause for the delay?

We're using Mach messages to pass packets from the SysExt to our XPC service (we are using exactly the same kind of messaging and code to pass from the root process to the XPC service). There's no other processing being done in the SysExt itself. Is there a faster IPC call we should be using to talk to the SysExt? Other options we've considered are passing a reference to a socket for direct communication or using a shared memory approach (mmap).

Are there any other common optimizations that we might need to investigate for the System Extension?

Our old design would drop outgoing packets if they weren't being processed fast enough (as we must grab them from the kernel either way). It's unclear what the SysExt does in this case - are they dropped, is there a buffer, could this be cause for the delay?

If you have read the packets from your interface I suspect that they are being buffered, or in a queue to be read. I am not aware if these packets are dropped at a certain point or not.

We're using Mach messages to pass packets from the SysExt to our XPC service (we are using exactly the same kind of messaging and code to pass from the root process to the XPC service). There's no other processing being done in the SysExt itself. Is there a faster IPC call we should be using to talk to the SysExt? Other options we've considered are passing a reference to a socket for direct communication or using a shared memory approach (mmap).

Under the hood an XPC service uses a socket so I do not think this will buy you any performance, and I suspect that you will run into sandboxing issues on the Network Extension side trying to do this as well.

I would open a bug report for the performance concerns you have outline here and ask about the dropped packets situation you have outlined.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Significant performance drop switching from Kernel to System Extension
 
 
Q