NETransparentProxyProvider doesn't invoke handleNewFlow

I am writing a network extension (activated from a MacOS app) which should proxy HTTP connections for certain domains through a custom made proxy server. This is the code I am playing with:

class AppProxyProvider: NETransparentProxyProvider {
    static let log = OSLog(subsystem: "com.example.myextension", category: "provider")

    override func startProxy(options: [String : Any]?, completionHandler: @escaping (Error?) -> Void) {
        let settings = NETransparentProxyNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
        let rule = NENetworkRule(destinationHost: NWHostEndpoint(hostname:"example.com", port:"443"), protocol: .TCP)
        settings.includedNetworkRules = [rule]

        self.setTunnelNetworkSettings(settings) { error in
            completionHandler(error)
        }
    }
    
    override func stopProxy(with reason: NEProviderStopReason, completionHandler: @escaping () -> Void) {
        completionHandler()
    }
    
    override func handleAppMessage(_ messageData: Data, completionHandler: ((Data?) -> Void)?) {
        if let handler = completionHandler {
            handler(messageData)
        }
    }
    
    override func sleep(completionHandler: @escaping () -> Void) {
        completionHandler()
    }
    
    override func wake() {
    }
    
    override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool {
        os_log(.debug, log: Self.log, "HANDLE NEW FLOW")

        return true
    }
}

I am able to successfully invoke the startProxy method which in turn registers the custom NETransparentProxyNetworkSettings and invokes the completionHandler without errors. I can see the network extension successfully enabled in the system.

But as soon as I try to make an HTTP request to the specified domain (example.com:443), the request hangs on the client and the handleNewFlow method of my network extension is never called so that I can proxy the traffic.

So my question is what is the correct way to setup a NETransparentProxyProvider (or a NEAppProxyProvider)?

My goal is to intercept connections to specific addresses and proxy them through a local TCP server (by wrapping them in HTTP CONNECT requests)

I also tried subclassing directly from NEAppProxyProvider and registering NETunnelNetworkSettings, but in this case the completion handler of setTunnelNetworkSettings is invoked with an error:

The operation couldn’t be completed. (NEAgentErrorDomain error 1.)

Here's my attempt with NEAppProxyProvider:

class AppProxyProvider: NEAppProxyProvider {
    static let log = OSLog(subsystem: "com.example.myextension", category: "provider")

    override func startProxy(options: [String : Any]?, completionHandler: @escaping (Error?) -> Void) {
        let settings = NETunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")

        self.setTunnelNetworkSettings(settings) { error in
            completionHandler(error)
        }
    }
    
    override func stopProxy(with reason: NEProviderStopReason, completionHandler: @escaping () -> Void) {
        completionHandler()
    }
    
    override func handleAppMessage(_ messageData: Data, completionHandler: ((Data?) -> Void)?) {
        if let handler = completionHandler {
            handler(messageData)
        }
    }
    
    override func sleep(completionHandler: @escaping () -> Void) {
        completionHandler()
    }
    
    override func wake() {
    }
    
    override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool {
        os_log(.debug, log: Self.log, "HANDLE NEW FLOW")

        return true
    }
}

You’ll find a bunch of hints and tips on this front in Debugging a Network Extension Provider. One key one is not to start out testing with Safari. Rather, configure your provider to claim a range of IP addresses and then use a simple tool — like nc or something you wrote yourself — to make a connection to that IP address. Once you’ve got that working, you can build things up from there.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks for the useful article, eskimo.

My extension works with Safari, but not with other applications.

I have configured my extension to intercept traffic to example.com:443:

let settings = NETransparentProxyNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
let rule = NENetworkRule(destinationHost: NWHostEndpoint(hostname:"example.com", port:"443"), protocol: .TCP)
settings.includedNetworkRules = [rule]

self.setTunnelNetworkSettings(settings) { error in
    completionHandler(error)
}

If I make the request with Safari then the handleNewFlow method is called.

But if I test with curl, the request hangs and the handleNewFlow method is not called (requests to other domains work with curl, so I suppose the interception rule is correct):

curl --http2 https://example.com -v
*   Trying 93.184.216.34:443...
* Connected to example.com (93.184.216.34) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):

Do I need some special settings in Info.plist or some other place to enable capturing from all apps?

Do I need .mobileconfig for Mac OS (currently I do not have any)?

Is the NENetworkRule that I configured sufficient to intercept network requests? I should also mention that the tunnelRemoteAddress that I specified (127.0.0.1) is fictional for the moment - once I get the handleNewFlow method triggered, I will implement the actual proxying part.

Let me repeat my previous advice: Rather than test with complex clients, like Safari and even curl, I recommend that you do your bring up using a client that you control.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I managed to make it work by tweaking the NENetworkRule passed to the settings and using a destination network.

Still wondering why NEAppProxyProvider with NETunnelNetworkSettings is throwing NEAgentErrorDomain error though. If I want to target Mac OS 10.5, I cannot use NETransparentProxyProvider:

class AppProxyProvider: NEAppProxyProvider {
    override func startProxy(options: [String : Any]?, completionHandler: @escaping (Error?) -> Void) {
        let settings = NETunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
        self.setTunnelNetworkSettings(settings) { error in
            completionHandler(error)
        }
    }
   ...
}
NETransparentProxyProvider doesn't invoke handleNewFlow
 
 
Q