How to divert traffic of specific applications to proxy that makes decision about termination of connection?

Question

Created Jun ’20

Replies 6

Boosts 0

Views 1.2k

Participants 2

Hi!

I am porting existing DPL system on to macOS, which have one key feature: ability to inspect the outgoing traffic of the specified applications (possibly all applications) and to skip the traffic of others not included in the list.

To distinguish traffic of required apps I am trying to use system network extension. Catched traffic should be diverted to the our application that acts like regular proxy which can make 2 desicion: block/terminate connection or allow it.

I was tried app-proxy(NEAppProxyProvider on sext side and NETransparentProxyManager on "client" side) type of network extension.
But I stuck with

Code Block @method handleNewFlow:

How can I divert traffic to my proxy? Documentation says call

Code Block -[NEAppProxyFlow openWithLocalEndpoint:completionHandler:]

on the flow and it's all? Will traffic goes through my proxy? For what purposes are exists methods like

Code Block -[NEAppProxyTCPFlow readDataWithCompletionHandler:completionHandler:]

or

Code Block -[NEAppProxyTCPFlow writeData:withCompletionHandler:]

?

My goal is get network extension that acts like transparent proxy for specified apps and diverts catched traffic to my existing proxy app, which allow or disallow connection.

One more: during experimentation with app-proxy NE I noticed that my VPN turned off when NEAppProxyProvider started. It is unacceptably, so I drew attention on content-filter(NEFilterDataProvider on sext side and NEFilterManager on "client" side) type of NE.

It seems that it works "before" all vpns and proxies which is desired functionality.

But again, I stuck: how organize diverting traffic to my proxy to make decision about connection?

As I understood general algorithm looks like:

Code Block handleNewFlow:(return [NEFilterNewFlowVerdict filterDataVerdictWithFilterInbound:NO peekInboundBytes:0 filterOutbound:YES peekOutboundBytes: ChunkBytesSize]; )	--> handleOutboundDataFromFlow:readBytesStartOffset:readBytes:

and here, in this method I should open connection to my proxy and write data. Am I correct?

I am sorry for so complex question, but I am really messed up with all these types of NE provides.

So, summarise: how to divert traffic of specified apps to my proxy that makes decision about termination of connection?

Boost

Answer 1

Systems Engineer OP

Apple

Jun ’20

Hello!

How can I divert traffic to my proxy?

For diverting a flow to be handled by your App Proxy you can make a decision about if you want to handle the flow and then return true from handleNewFlow or false to not handle the flow.

Code Block swiftoverride func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool {
	
	// Make a decision on whether handle the flow. Return true to handle and retain the flow.
	// Return false to not handle the flow.
}

Documentation says call on openWithLocalEndpoint on the flow and it's all? Will traffic goes through my proxy?

Opening the new flow tells the proxy that it is ready to read/write data to and from that flow.

For what purposes are exists methods like -[NEAppProxyTCPFlow readDataWithCompletionHandler:completionHandler:] or -[NEAppProxyTCPFlow writeData:withCompletionHandler:]

The read and write methods for NEAppProxyTCPFlow are methods that would be used when data is read from your flow and written to your new remote connection. Likewise for reading from your remote connection and written to your flow.

My goal is get network extension that acts like transparent proxy for specified apps and diverts catched traffic to my existing proxy app, which allow or disallow connection.

You are on the right track with the mentioned methods above. For a remote connection take a look at using NWConnection or NWTCPConnection.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

0

Answer 2

alex_gtb OP

Jun ’20

Hello, Matt

Thank you for response.

I did some experimentations, but didn't reach success.

just to clarity, my goal looks like(as I can imagine):

Code Block any_app <----> network extension(based on NEAppProxyProvider) aka transparent proxy < --- > my_smart_proxy < --- > internet

i.e. ne sext catches traffic any application, forward it to mysmartproxy, the last one determines if there is sensitive data and blocks the connection if true or allows otherwise - just pass all in- and out-data from any app to internet and vice versa.

So, my steps:

in - (void)startProxyWithOptions: completionHandler: I've opened connection to my mysmartproxy in that way:

Code Block languageconn = [ self createTCPConnectionToEndpoint:[ NWHostEndpoint endpointWithHostname:@"127.0.0.1" port:@"11234"]
                            enableTLS:NO
                          TLSParameters:nil
                            delegate:nil];

in handleNewFlow: I did [ flow openWithLocalEndpoint:nil completionHandler: ], after that in completionHandler perform [tcpFlow readDataWithCompletionHandler:], in appropriate handler I tried to write recieved data to my mysmartproxy via [ self->conn write:data completionHandler:]. Every time error is occurs:

Code Block 2020-06-25 14:31:13.281 com.gtb.my_ext[52343:1028084] NWTCPConnection:write error: 'The operation couldn’t be completed. (kNWErrorDomainPOSIX error 57 - Socket is not connected)'

and indeed - there is no any connection to my proxy, I can confirm it looking in mysmartproxy's logs.

So, what is wrong?

just in case: I've configured sext with disabled sandbox.

0

Answer 3

alex_gtb OP

Jun ’20

And one more: even if connection to proxy was successful and [ self->conn write:data completionHandler:] performs without error how can I receive response from proxy and put it back to the flow(via -[NEAppProxyTCPFlow writeData:withCompletionHandler:])?

I saw

Code Block -(void)readLength:completionHandler:

and

Code Block -(void)readMinimumLength:maximumLength:completionHandler:

methods in NWTCPConnection class, but I don't understand how can specify required length? 'cause in general I don't know size of response from any site/service in internet.

0

Answer 4

Systems Engineer OP

Apple

Jun ’20

2020-06-25 14:31:13.281 com.gtb.my_ext[52343:1028084] NWTCPConnection:write error: 'The
operation couldn’t be completed. (kNWErrorDomainPOSIX error 57 - Socket is not connected)'

The write error suggests that the NWTCPConnection was not ready yet. Now this could be due to the connection not going into the ready state or the connection attempt resulted in an error. I am betting it is the first case because you would have seen an immediate error on the second. I would recommend setting up an observer to monitor the connection status using NWTCPConnectionState. Make sure the connection goes into NWTCPConnectionStateConnected before write data to the connection.

From here, the workflow I would use is to open the connection to the smart proxy and then open your TCP flow received from your App Proxy Provider.

And one more: even if connection to proxy was successful and [ self->conn write:data completionHandler:] performs without error how can I receive response from proxy and put it back to the flow(via -[NEAppProxyTCPFlow writeData:withCompletionHandler:])? I don't understand how can specify required length? 'cause in general I don't know size of response from any site/service in internet.

This is up to you based on how you want to handle the incoming data. Understandably for all traffic you do not know a suitable length to read a lower and upper bound from using readMinimumLength / maximumLength, so you could set this to something between 2 and 65,535 bytes and your will receive data between that range for each connection. This will give you essentially whatever is available. If you are targeting specific connections that contain specific frames, and you need to read a certain length each time, this is where I could see readLength coming in handy.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

0

Answer 5

alex_gtb OP

Jun ’20

I would recommend setting up an observer to monitor the connection status using NWTCPConnectionState. Make sure the connection goes into NWTCPConnectionStateConnected before write data to the connection

I did it, connection hangs in NWTCPConnectionStateConnecting status forever.

Is any way how to understand reason of such behaviour?

0

Answer 6

Systems Engineer OP

Apple

Jun ’20

This explains why you could not write to connection earlier. Have you tried connecting to a known address that you know you will be able to open a connection to instead of 127.0.0.1? What this will prove is whether you need to debug listening for connections at your smart proxy or if the issue is in your App Proxy logic somewhere.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

0