Ordering of Transparent Proxy Providers

App & System Services Networking
Eric K. OP
Created Apr ’21
Replies 11
Boosts 0
Views 1.3k
Participants 4

Observation:

We observed that when multiple Transparent Proxies (NETransparentProxyProvider) are active and enabled, only one will handle the flow, and each proxy is given an opportunity to handleNewFlow() in a strict ordering.

We also observed that any connections (NWConnection) created by a transparent proxy will not be intercepted by other transparent proxies eariler in the ordering.

We further observed that this ordering is dependent on the ordering in which the Network Extensions are installed.


Question:

1. Is there a way to control the order in which each transparent proxy has an opportunity to handle the flow?

2. Is the following observed scenario correct or expected?


Observed Scenario:

Safari --> Transparent Proxy Provider A --> Transparent Proxy Provider B --> Destination (Web)

When a connection is created by Safari:

  • Proxy A is called with handleNewFlow()

    • if Proxy A decides to handle the flow, then Proxy B would not be called for this flow

    • if Proxy A decides not to handle the flow, then Proxy B would be called with handleNewFlow()

When a connection is created by Proxy A:

  • Proxy B will be called with handleNewFlow() for Proxy A's connection.

When a connection is created by Proxy B:

  • Proxy A would not be called with handleNewFlow() for Proxy B's connection.


Context:

In general, security software would want to be first in the ordering while VPN software would want to be last in the case where both VPN and security software are installed.
Replies  11
Boosts  0
Views  1.3k
Participants  4
Systems Engineer OP
Apple
Apr ’21

  1. Is there a way to control the order in which each transparent proxy has an

opportunity to handle the flow?
No, not that I am aware of.

2. Is the following observed scenario correct or expected?

I can only offer an opinion here based on the results of testing that I have done;
I enabled two NETransparentProxyProvider's that will handle TCP connections from Safari. NETransparentProxyProvider One and NETransparentProxyProvider Two. When a Safari flow was initiated, NETransparentProxyProvider One picked it up and proxied the flow, while NETransparentProxyProvider Two did see the flow but did not recognize the flow as a Safari flow that needed to be proxied.

Code Block text
NETransparentProxyProvider One:
2021-04-06 07:24:00.936311-0700 0x81a7     provider received a new flow, flow: 0x7fc5aa71a4a0 TCP com.apple.Safari[{length = 20, bytes = ***}] remote: x.x.x.x:443 interface en0
2021-04-06 07:24:00.939693-0700 0x81a7     new flow with bundle id com.apple.Safari is being claimed
2021-04-06 07:24:00.939728-0700 0x81a7     NEAppProxyFlow, flow: 0x7fc5aa71a4a0 - (WILL be claimed) by the proxy.
2021-04-06 07:24:00.939768-0700 0x81a7     Flow Provider will handle new NEAppProxyTCPFlow, flow: 0x7fc5aa71a4a0 - TCP com.apple.Safari[{length = 20, bytes = ***}] remote: x.x.x.x:443 interface en0
2021-04-06 07:24:00.939830-0700 0x81a7     provider core will start new TCP flow, flow: 0x7fc5aa71a4a0
2021-04-06 07:24:00.939877-0700 0x81a7     provider core will start new TCP flow flow.remoteEndpoint.nwEndpoint.debugDescription x.x.x.x:443, flow: 0x7fc5aa71a4a0
2021-04-06 07:24:00.939945-0700 0x81a7     provider core did start TCP flow, copier: 80, flow: 0x7fc5aa71a4a0
2021-04-06 07:24:00.953431-0700 0x82eb      # Copier - Ready 80 #



Code Block text
NETransparentProxyProvider Two:
2021-04-06 07:24:01.006642-0700 0x7948     provider received a new flow, flow: 0x7f87fbd12e80 TCP com.example.apple-samplecode.TransparentProxyTestBed.TransparentProxy-1[{length = 20, bytes = ***}] remote: x.x.x.x:443 interface en0
2021-04-06 07:24:01.009693-0700 0x7948     NEAppProxyFlow, flow: 0x7f87fbd12e80 - (WILL NOT be claimed) by the proxy.
2021-04-06 07:24:01.009722-0700 0x7948     Operating System will handle new flow: 0x7f87fbd12e80 - (null)


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
Eric K. OP
Apr ’21
@meaton, thank you very much for your reply.

As a followup, is there a way for NETransparentProxyProvider Two to identify the process which the flow is originated from?

Should we raise a Feedback in order to achive this?
0
Systems Engineer OP
Apple
Apr ’21

As a followup, is there a way for NETransparentProxyProvider Two to identify the process which the flow is originated from?

Good question. I would check this first with a Distribution signed copy of the App / Extension. During my tests I was testing with SecCodeCopySigningInformation to get the source information of the flow based on the signature of the app that generated the flow. In this case NETransparentProxyProvider One was signed with a Development signature and so not all of the information may have been available in the dictionary returned from SecCodeCopySigningInformation. If you have not, I would test this condition first before opening a bug report.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
Eric K. OP
Apr ’21
Identifing the process which the flow is originated from in NETransparentProxyProvider One shouldn't be a problem.

It's the scenario where NETransparentProxyProvider Two is only seeing network traffic generated from NETransparentProxyProvider One. Is there a way for NETransparentProxyProvider Two to know the true source of the flow? As far as we can tell now, NETransparentProxyProvider Two will identify all incoming flow to have been generated from the process of NETransparentProxyProvider One.
0
Systems Engineer OP
Apple
Apr ’21

Is there a way for NETransparentProxyProvider Two to know the true source of the flow? As far as we can tell now, NETransparentProxyProvider Two will identify all incoming flow to have been generated from the process of NETransparentProxyProvider One.

How are you identifying the source of the flow? When you try to identify the source of the flow from NETransparentProxyProvider Two, does it provide you the signature information of NETransparentProxyProvider One, providing that both providers are using a Developer ID signature and NETransparentProxyProvider One is handling the flow?

To answer your question though, does using setMetaData on NEAppProxyFlow expose anything new to NETransparentProxyProvider Two if NETransparentProxyProvider One is handling the flow?


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
Eric K. OP
Apr ’21
Our currently sitituation is that when our product is NETransparentProxyProvider Two and another vendor is NETransparentProxyProvider One, we are unable to identify the originating process of the flow content.

We curerntly use the flow.metadata to identify the originating process as a browser. However, we do not have that ability being NETransparentProxyProvider Two as flow.metadata reflects process information of NETransparentProxyProvider One.

Is there anyway of being NETransparentProxyProvider Two and knowing the originating process information that was intercepted by NETransparentProxyProvider One?
0
Systems Engineer OP
Apple
Apr ’21

Is there anyway of being NETransparentProxyProvider Two and knowing the originating process information that was intercepted by NETransparentProxyProvider One?

You may be running into a limitation here or a bug. I am not sure. To figure out more, open a TSI and either Quinn or myself can look into this more.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
Will_32768 OP
Jun ’21

Is there any tool or API that we can use to determine what order the NETransparentProxyProvider system extensions will be called with handleNewFlow?

0
Systems Engineer OP
Apple
Jun ’21

@Will_32768:

Is there any tool or API that we can use to determine what order the NETransparentProxyProvider system extensions will be called with handleNewFlow?

To my knowledge, no.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
JonoO OP
Aug ’24

Thank you Apple for implementing the Order property as per https://developer.apple.com/documentation/devicemanagement/vpn/transparentprox. This is designed to solve the problem described above. Please could we have some help implementing a test profile to validate this behaviour. Is there a way to validate that the Order property is set, and working as expected? Thanks Jono

0
JonoO OP
Aug ’24

Correcting in the URL above - https://developer.apple.com/documentation/devicemanagement/vpn/transparentproxy

0
Ordering of Transparent Proxy Providers
First post date Last post date
 
 
Q