Transparent Proxy / AppProxyProvider can't see new flows?

I work for a company that is replacing their current endpoint security product that uses an NKE implementation for something that will work in future macOS versions. We need to capture various metadata about connections (ip/port, etc.) and also to possibly divert web traffic to a remediation/capture page for dangerous or unwanted (in the IT sense) browser traffic.


Since the content filter APIs won't divert traffic (from what I've read), I have started looking into the Transparent Proxy APIs. I have taken the SimpleFirewall example and replaced the content filtering calls with transparent proxy calls. At this point, my goal is to prove I can get notified of new flows, get metadata about those flows, and eventually see data from those flows.


This is how I am creating the AppProxyProvider:

override func startProxy(options: [String : Any]? = nil, completionHandler: @escaping (Error?) -> Void) {
       
        let includeRule = NENetworkRule(destinationHost: NWHostEndpoint(hostname: "sophostest.com", port: "80"), protocol: .TCP)
        
        let proxySettings = NETransparentProxyNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
        proxySettings.includedNetworkRules = [includeRule]
        
        setTunnelNetworkSettings(proxySettings) { error in
            if let applyError = error {
                os_log("Failed to apply proxy settings: %{public}@", log:.networkExtension, applyError.localizedDescription)
            }
            completionHandler(error)
        }
    }

To make things simple, I am just looking at one sites traffic on one port, as to not overwhelm the logging/developer. 🙂 My main area of concern with this code is the creation of the NETransparentProxyNetworkSettings object. If I don't supply a tunnelRemoteAddress or if I try to supply a port, it doesn't "connect" and I get this error in the console

Failed to apply proxy settings: Missing NETunnelNetworkSettings tunnelRemoteAddress

At this point, I am not trying to proxy the data at all. I am assuming that the app would be able to view the traffic before deciding if it was going to proxy it.


For some reason I have to go into the network control panel to enable the connection (like a VPN). I'm not sure why this needs to happen, and it is something I will need to automate eventually. I can't rely on the user to do this.


My handleNewFlow looks like this

    override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool {
        
        if let tcpflow = flow as? NEAppProxyTCPFlow
        {
            os_log("tcpflow handled - %{public}@",log:.networkExtension, tcpflow.metaData.debugDescription)
        }
        else if let udpflow = flow as? NEAppProxyUDPFlow
        {
            os_log("udpflow handled - %{public}@", log:.networkExtension, udpflow.metaData.debugDescription)
        }
        else
        {
            os_log("flow handled - %{public}@",log:.networkExtension, flow.metaData.debugDescription)
        }
        return false
    }


As you can see if this method is called, something should be sent to os_log. When I browse to sophostest.com with the transparent proxy started and connected (via the network control panel), I get an "operation not permitted" page in safari (understandable) and no os_log messages at all.

If I don't supply a

tunnelRemoteAddress
… it doesn't "connect" and I get this error in the console

Yeah, that’s a bit weird. The requirement to populate the

serverAddress
property of the
NETunnelProviderProtocol
object you use to create the configuration and the
tunnelRemoteAddress
property of the
NETransparentProxyNetworkSettings
object you use to start the tunnel is inherited from the VPN roots of this technology. For a transparent proxy, it’s fine to use
localhost
and 127.0.0.1 respectively, unless you have something more sensible.

For some reason I have to go into the network control panel to enable the connection (like a VPN).

Enable? Or connect? If you need to enable it, it sounds like you’re not setting the

isEnabled
property of the
NETransparentProxyManager
when you save you configuration. OTOH, if you talking about connecting it, yeah, starting the connection is something you need to do. You can do it programmatically via the
connection
property of the
NETransparentProxyManager
. As the transparent proxy is treated as a tunnel, you can cast that to
NETunnelProviderSession
.

As you can see if this method is called, something should be sent to

os_log
. When I browse … I get an "operation not permitted" page in Safari (understandable) and no
os_log
messages at all.

I’m not sure what’s going on there. I have something that might help you debug this, but I can’t post it here. Please drop me a line via email (my address is in my signature, below).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

ps DTS is closed 25…29 Nov in observance of the US Thanksgiving holiday.

Hi guys,


I tried same thing and faced similar issue. Could you help on how it is solved??

I tried starting a new discussion but it is stuck at "waiting on moderator approval" from last couple of days.

Transparent Proxy / AppProxyProvider can't see new flows?
 
 
Q