NEAppProxyProvider with dynamic rules

App & System Services Networking
SerusMac OP
Created Oct ’20
Replies 12
Boosts 0
Views 1.5k
Participants 3
Our product used NKE to proxy a limited set of selected programs and connections. Now, we are rewriting it to NEAppProxyProvider. It seems that this API has a major problem. We have not found a way to proxy only selected programs and/or make filtering decisions dynamically.
It seems like the rules for NEAppProxyProvider can only be set on VPN start and this does not support per-program rules at all.
Is this correct?

This is especially strange because NEFilterDataProvider (no traffic modification) can work with NEFilterNewFlowVerdict which allows to make flow decisions dynamically. In contrast, handleNewFlow of NEAppProxyProvider returns BOOL (reject or process). So, we have to process all flows. And if a flow has to stay intact (the vast majority of cases) we have to manually replicate it. Effectively, this forces all connections on the system to go through our app while we need only a few. This has a very bad impact on the network performance and compatibility with other products (coexistence is impossible).

So is there a way to work with only a limited set of flows with NEAppProxyProvider like with NEFilterDataProvider?
Answered by SerusMac in 641730022
Thank you very much for this hint!
We are still testing but it seems that this update solves all the problems.

By the way, this change was not rendered in the documentation.
We have only found this in the source code comments.
Replies  12
Boosts  0
Views  1.5k
Participants  3
Systems Engineer OP
Apple
Oct ’20

We have not found a way to proxy only selected programs and/or make filtering decisions dynamically.

Take a look at the new NETransparentProxyProvider API in macOS 11 Beta 10 as this does provide some flexibility here.

Most notably:

Returning NO from handleNewFlow: and handleNewUDPFlow:initialRemoteEndpoint: causes the flow to proceed to communicate directly with the flow's ultimate destination, instead of closing the flow with a "Connection Refused" error.

So the workflow could be to filter on all traffic, decide if the application is coming from an app you want to filter and return true to handle the filter. Return false to let the flow traverse the system as normal.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
SerusMac OP
Oct ’20
Accepted Answer
Thank you very much for this hint!
We are still testing but it seems that this update solves all the problems.

By the way, this change was not rendered in the documentation.
We have only found this in the source code comments.
0
Systems Engineer OP
Apple
Oct ’20

Thank you very much for this hint!
We are still testing but it seems that this update solves all the problems.

No problem. Glad this is working out for you. This looks like an excellent addition to the Proxy Provider APIs.

By the way, this change was not rendered in the documentation.
We have only found this in the source code comments.

Right. I did notice that as well. This would be an opportunity to open an enhancement request to further document this. Please respond back with the feedback ID.

<https://developer.apple.com/bug-reporting/>


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
SerusMac OP
Oct ’20

Right. I did notice that as well. This would be an opportunity to open an enhancement request to further document this. Please respond back with the feedback ID.

Done. Feedback ID is FB8829962.


0
Systems Engineer OP
Apple
Oct ’20
Thank you, I see the bug report internally and have requested access for further updates.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
vmku OP
Apr ’21
Hi Matt,

So the workflow could be to filter on all traffic, decide if the application is coming from an app you want to filter and return true to handle the filter. Return false to let the flow traverse the system as normal.

As mentioned in the above, in NETransparentProxyProvider, Returning NO from handleNewFlow: lets the flow traverse the system as normal, but we have a requirement where we want to stop flow traversing the system as normal in this case (achieving something similar to "Connection Refused" in NEAppProxyProvider). Any suggestions or workarounds will be very helpful here. Thanks
0
Systems Engineer OP
Apple
Apr ’21
@vmku

but we have a requirement where we want to stop flow traversing the system as normal in this case (achieving something similar to "Connection Refused" in NEAppProxyProvider). Any suggestions or workarounds will be very helpful here. Thanks

For NETransparentProxyProvider discarding the flow would be an enhancement request.
You could try doing content filtering with NEFilterDataProvider in front of the proxy as an option.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
vmku OP
May ’21

For NETransparentProxyProvider discarding the flow would be an enhancement request.
You could try doing content filtering with NEFilterDataProvider in front of the proxy as an option.

Thanks Matt, opened FB9099075 for it.
0
Systems Engineer OP
Apple
May ’21
Thank you @vmku. To workaround this, you could look at the following options:

  1. Use NEAppProxyProvider instead of NETransparentProxyProvider.

  2. Return YES from handleNewFlow: and then immediately call closeReadWithError: and closeWriteWithError: on the flow.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

0
vmku OP
May ’21
Thanks Matt,

Use NEAppProxyProvider instead of NETransparentProxyProvider.

Couldn't get above completely, NETransparentProxyProvider extends NEAppProxyProvider and NETransparentProxyProvider available in macOS 11.0 (BigSur). We have extended NETransparentProxyManager in our TransparentProxy, and receiving flow at handleNewFlow:


0
Systems Engineer OP
Apple
May ’21

We have extended NETransparentProxyManager in our TransparentProxy, and receiving flow at handleNewFlow:

No problem. In that case you could try the approach of: Return YES from handleNewFlow: and then immediately call closeReadWithError: and closeWriteWithError: on the flow.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
vmku OP
May ’21

No problem. In that case you could try the approach of: Return YES from handleNewFlow: and then immediately call closeReadWithError: and closeWriteWithError: on the flow.


Thanks much Matt, it worked on Catalina (macOS 10.15.7). Having inherent issue on BigSur(macOS 11) with NETransparentProxyProvider and so couldn't try it on BigSur, will open DTS for this.

0
NEAppProxyProvider with dynamic rules
First post date Last post date
 
 
Q