Transparent proxy on macOS is unable to catch accountsd connections

App & System Services Core OS
rg1985 OP
Created Dec ’20
Replies 5
Boosts 0
Views 2.4k
Participants 2
Hi,

We are working on developing a network transparent proxy using system extension for macOS Big Sur using NEAppProxyProvider and NETransparentProxyManager APIs for capturing SMTP, IMAP and POP connections.

We have defined network rules in proxy provider to capture outgoing connections on ports 25, 465, 143, 993. We defined only remote network with wild card IPv4 address "0.0.0.0" and above ports.

It is able to catch the outgoing connection to port 993 from Mail.app.

But when we try to change mail account setting to port 143 and save, Mail app was not making any connection. With some analysis, we found that accountsd was handling accounts and when we change any setting such as port and save, accountsd was connecting to mail server on that port.

Now the strange behaviour is our transparent proxy is not able to catch this accountsd connection to port 143.

If we use old good NKE on 10.15, we are able to capture outgoing connection from accountsd process. We expect similar behaviour with network system extension too.

Right now, we are stuck on this issue.

How can we make our network system extension be able to catch accountsd traffic?

Thanks in advance!!




Replies  5
Boosts  0
Views  2.4k
Participants  2
Systems Engineer OP
Apple
Dec ’20

Now the strange behaviour is our transparent proxy is not able to catch connection to port 143.

Okay, this is strange that it would work for 993 and not 143. Can you post your NENetworkRule here. Do you see this get to at least handleNewFlow and you cannot proxy the connection?


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
rg1985 OP
Dec ’20
Thanks Matt for the reply!

When we add/update IMAP account settings and save, then the flow is going through accountsd (either 993 or 143) which can not be captured by transparent proxy.

Once the mail account is configured and saved (either it is 143 or 993), if we send a mail from Mail.app, then Mail app is making connection to Mail Server and that is being captured.

So in brief, Mail.app makes connection (any SMTP or IMAP ports) => proxy catches that flow.

Steps to reproduce the specific scenario are:
#1: Add an IMAP mail account on IMAP SSL port 993 and SMTP non-SSL port 25 and save it.
#2: Install transparent proxy and it is running with the configured network rules to catch any SMTP and IMAP traffic.
#3. Send mail from Mail.app and it gets captured.
#4: While transparent proxy is up and running, open IMAP mail account settings and edit settings of IMAP from SSL to nonSSL and port from 993 to 143 and click Save. When Save is clicked, the connection which goes out is from accountsd and not from Mail.app. This is not being captured.

My network rules are as below:

  • (NENetworkRule *)createNeRuleFor:(NSString *)port {

[[NENetworkRule alloc] initWithRemoteNetwork:[NWHostEndpoint endpointWithHostname:@"0.0.0.0" port:port]
                                           remotePrefix:0
                                           localNetwork:nil
                                           localPrefix:0
                                           protocol:NENetworkRuleProtocolTCP
                                           direction:NETrafficDirectionOutbound];
}
settings.includedNetworkRules = @[
        [self createNeRuleFor:@"25"],
        [self createNeRuleFor:@"587"],
        [self createNeRuleFor:@"110"],
        [self createNeRuleFor:@"995"],
        [self createNeRuleFor:@"143"],
        [self createNeRuleFor:@"993"],
    ];

Query#1: Is there any restriction on the ability of network transparent proxies on catching network traffic from some system processes such as accountsd process?




0
Systems Engineer OP
Apple
Dec ’20
Short answer is that I'm not sure why that does not work mid-stream or if the accountsd traffic is not being proxied.

If you start with those ports in your NETransparentProxyNetworkSettings then I was at least able to capture 993 and 587.

Code Block text
2020-12-03 07:18:13.860621-0800 0x11cdc1   Debug       0x0  24510  0    provider received a new flow, flow: 0x7f9e82809b80 TCP com.apple.mail[{length = 20, bytes = *}] remote: x.x.x.x:993 interface en0
2020-12-03 07:18:13.867991-0800 0x11cdc1   Debug       0x0  24510  0    new flow with bundle id com.apple.mail is being claimed
2020-12-03 07:18:13.868025-0800 0x11cdc1   Debug       0x0  24510  0    NEAppProxyFlow, flow: * - (WILL be claimed) by the proxy.
2020-12-03 07:18:41.296939-0800 0x11cf29   Debug       0x0  24510  0    provider received a new flow, flow: 0x7f9e81704c70 TCP com.apple.mail[{length = 20, bytes = *}] remote: x.x.x.x:587 interface en0
2020-12-03 07:18:41.300108-0800 0x11cf29   Debug       0x0  24510  0    new flow with bundle id com.apple.mail is being claimed
2020-12-03 07:18:41.300139-0800 0x11cf29   Debug       0x0   24510  0    NEAppProxyFlow, flow: * - (WILL be claimed) by the proxy.


This was tested with an IMAP account checking new mail and sending outgoing mail.
I realize this does not exactly cover the scenario you are describing so for all other inquiries on this I recommend that you open a bug report.. Please respond with the Feedback ID.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
rg1985 OP
Dec ’20
Opened feedback request, ID is FB8935382
0
Systems Engineer OP
Apple
Dec ’20
Thank you for opening a bug report. I see it internally. I suspect it would be helpful to add a sysdiagnose with the VPN debugging and Network Diagnostics profile installed as well. See instructions on how to do this here.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
0
Transparent proxy on macOS is unable to catch accountsd connections
First post date Last post date
 
 
Q