Transparent Proxy seems to break Mail.app on Big Sur

You mentioned:

(I've attached an example to feedback FB8690357).

I found this internally and do not see an attached focused sample project? Could you try uploading it again?


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Greetings, @meaton!

I've put the code on github: https://github.com/ngorskikh/transparentproxyrepro

I've updated the feedback, too, but it looks like attachments don't work for some reason -- or at least I can't see my own attachments, even though the original feedback request contained some automatically collected logs, as well as a zip with an xcode project and a couple of screenshots.

Kind regards, ngorskikh.

It turns out, attachments in Feedback Assistant are also affected by the presence of a transparent proxy network extension.
I've updated the feedback once more, now my data seems to be successfully attached.

Thanks for adding the latest information. To be honest this report seemed odd to me from a general mail usage case, so I took a few minutes verified that the Mail.app does still receive email when a Transparent Proxy is running with a very broad set of TCP/UDP NENetworkRules rules. For example:

Code Block swift
settings.includedNetworkRules = [
    NENetworkRule(remoteNetwork: nil, remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol: .TCP, direction: .outbound),
    NENetworkRule(remoteNetwork: nil, remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol: .UDP, direction: .outbound)
]

When running this on macOS 11 Big Sur (Big Sur 11 beta 6) and built with Xcode 12, I was able to start the Transparent Proxy as normal and see my flow copier start running. I then sent an email to my inbox that I configured just for this test and it was received just fine. I seen the email come through my flow copier and I could view the email contents in the Mail.app with no problem.

Code Block text
2020-09-16 07:26:13.503985-0700 0x295890   remote: imap.gmail.com:993
2020-09-16 07:26:13.507324-0700 0x295890   new flow with bundle id com.apple.mail
2020-09-16 07:26:13.507378-0700 0x295890   ### (TCP) ### provider will handle new NEAppProxyTCPFlow, flow: 0x7fe04261a4d0 - TCP com.apple.mail[{length = 20, bytes = ...}] remote: imap.x.com:993
2020-09-16 07:26:13.544090-0700 0x29592c   copier 19 connection did change state,  
2020-09-16 07:26:13.544120-0700 0x29592c   ##### Copier - Ready 19 #####
2020-09-16 07:26:13.544133-0700 0x29592c   copier 19 connection did change state, state: ready

Now, there are a few distinct differences about my test and the test that you ran, ngorskikh. Here are those differences:

1) I did not disable IPv6 on my network interface.
2) I did not use a Yahoo account when testing.
3) My flow copier is not setup to not use nil to open the flow and the flow is open only after the remote side of the connection is opened.

In regards to number 3, in my testing I alway open the remote side of the connection first so that the flow can be opened with connection.currentPath?.localEndpoint, instead of using nil. Why do you use nil to open the flow here for the localEndpoint? I would try rearranging your flow copier to open your flow ONLY if you have a localEndpoint on your localPath available. Note that this will involve rewriting your flow copier. Let me know if this improves your situation. If not, open a TSI and I can dig in further.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Thanks for looking into this!

I've updated the feedback with a revised sample that opens the flow after outbound connection has been established, and sets the local endpoint accordingly. Unfortunately, this doesn't seem to help. I've also reproduced an issue with Safari seeing 0.0.0.0 and how it breaks Youtube, and attached some screenshots.

Please note that disabling IPv6 is crucial to reproducing this bug, and also the issue itself is not with receiving emails, but just with trying to add a Yahoo account in Mail. And, of course, there's also the newly uncovered issue with attachments not working in Feedback Assistant, when the same network extension is active.

By the way, I was under the impression that such broad rules as in your example are prohibited? I'm referring to these statements from the documentation (https://developer.apple.com/documentation/networkextension/netransparentproxynetworksettings/3143656-includednetworkrules?language=objc):

If the port string of the endpoint is 0 or is the empty string, then the address of the endpoint must be a non-wildcard address, such as 0.0.0.0 or ::.

If the address is a wildcard address (such as 0.0.0.0 or ::), then the port string of the endpoint must be non-empty and must not be 0.

By the way, I've updated the feedback with all the requested information and all the additional issues that I've encountered so far with hopefully more clear steps to reproduce, and even a screen cast. Please take a look.

Thank you for adding the additional information. I can assure you that this landed in the correct hands internally, so if there are further updates to this very specific case, they will be added to the bug report.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

The issue is still here on Beta 9.

so if there are further updates to this very specific case

I am quite surprised that this case is considered "very specific". All users without IPv6 are affected. As far as I know, there are way more users like that than others.

Also, I suppose this thread should've been called "Transparent Proxy breaks Youtube for all users without IPv6" to make it clear that this whole thread is not about a specific case of breaking IMAP, but about a much more serious issue.

Is there at least any workaround for that?

Please retry this with the update to macOS Big Sur Beta 10.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Hi! I've retried on Big Sur beta 10 and everything still reproduces, unfortunately (all remote hosts seen as 0.0.0.0 in Safari and SSH is particularly depressing)

I've also encountered a new problem, should I open a separate bug report? The problem is as follows:

1. Configure a HTTP and HTTPS proxy in system settings for your network interface. The proxy must be in your local network, NOT on a localhost.

2. Now start a transparent proxy extension.

3. Open any site in Safari.

4. Transparent proxy receives a flow with the site's domain name and proxy's port as the remote endpoint.

5. Obviously, traffic is not routed through system-configured proxy, and the port is wrong, so Internet completely breaks :(

I'll attach some clarifying screenshots to the feedback.

Please make sure that you are testing with the new NETransparentProxyProvider in macOS 11 Beta 10. This is a new provider class in Beta 10. I ran some tests this morning and you can start testing by changing NEAppProxyProvider for NETransparentProxyProvider.

Please test with this specific provider and update your bug report.

I've also encountered a new problem, should I open a separate bug report?

The following issue that you have described can be fixed in Catalina with the following settings. Please make sure you pay attention to the documentation notes when you test this with NETransparentProxyProvider:

Obviously, traffic is not routed through system-configured proxy, and the port is wrong, so Internet completely breaks :(

Can be resolved by adopting NEProxySettings() and setting it to the NETransparentProxyNetworkSettings. A very brief example of this might look like:

Code Block swift
let proxySettings = NEProxySettings()
guard let systemConf = SCDynamicStoreCopyProxies(nil) as? [CFString: AnyObject] else {
    return nil
}
proxySettings.autoProxyConfigurationEnabled = systemConf[kSCPropNetProxiesProxyAutoConfigEnable] as? Bool ?? false
proxySettings.proxyAutoConfigurationURL = systemConf[kSCPropNetProxiesProxyAutoConfigURLString] as? String
proxySettings.proxyAutoConfigurationJavaScript = systemConf[kSCPropNetProxiesProxyAutoConfigJavaScript] as? String

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

@meaton, Hi! I'm trying to run my app after changing the base class of my provider from NEAppProxyProvider to NETransparentProxyProvider and I'm getting a SIGCONT here:

Code Block 
Thread 1#0	0x0000000118c23eb4 in ImageLoaderMachOCompressed::updateOptimizedLazyPointers(ImageLoader::LinkContext const&) ()
#1	0x0000000118c22720 in ImageLoaderMachOCompressed::doBind(ImageLoader::LinkContext const&, bool, ImageLoader const*) ()
#2	0x0000000118c15b71 in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#3	0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#4	0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#5	0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#6	0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#7	0x0000000118c07955 in dyld::_main(macho_header const*, unsigned long, int, char const, char const, char const**, unsigned long*) ()
#8	0x0000000118c0022b in dyldbootstrap::start(dyld3::MachOLoaded const*, int, char const**, dyld3::MachOLoaded const*, unsigned long*) ()
#9	0x0000000118c00025 in _dyld_start ()

After typing continue in the debugger, the program crashes. Can you please help me with this? The documentation for NETransparentProxyProvider is currently blank.
I'm using Xcode Version 12.2 beta 3 (12B5035g), should I be using something else?

Hmm, it seems that now even after going back to NEAppProxyProvider the app won't start.
Maybe it's not related to the change, and instead an issue with the latest xcode beta.

@meaton Regarding proxy issues: I've tried your suggestion to set the proxySettings property of NETransparentProxyNetworkSettings. I've tried something like this (just to test if it will help)

Code Block 
NETransparentProxyNetworkSettings *sets = [[NETransparentProxyNetworkSettings alloc] initWithTunnelRemoteAddress:@"127.0.0.1"];
sets.proxySettings = [[NEProxySettings alloc] init];
sets.proxySettings.HTTPEnabled = YES;
sets.proxySettings.HTTPServer = [[NEProxyServer alloc] initWithAddress: @"192.168.1.11" port:12345];
sets.proxySettings.HTTPSEnabled = YES;
sets.proxySettings.HTTPSServer = [[NEProxyServer alloc] initWithAddress: @"192.168.1.11" port:12345];

The behaviour is identical to when the proxy settings are just set through the system settings: my app receives a flow with remoteEndpoint set to <website hostname>:<proxy port>, like this:

Code Block 
TCP com.apple.CalendarAgent[{length = 20, bytes = 0xefd63d533d62ec86c8d6149c2bf2699b909a37ac}] remote: p45-caldav.icloud.com:12345

I don't see how I can make a connection when I receive a wrong hostname:port pair. I might have misunderstood your suggested solution, would you please elaborate?

Deleted, last answer by topic starter is more informative description of that problem persists.

@ngorskikh

In regards to the proxy issue. Here is how I've successfully tested this in Catalina:

1) Set the Auto Proxy Discovery on the network interface in System Preferences.
2) Set autoProxyConfigurationEnabled as true and for testing purposes set a URL to a PAC in proxyAutoConfigurationURL.
3) In the PAC make sure you are setting PROXY results.
4) When flows comes through in handleNewFlow that match the PAC, note that these flows with get the proxied IPs.
5) Note that this has been tested in Catalina and the new provider for NETransparentProxyProvider may ignore these settings in Big Sur.

In regards to your crash it looks like you are seeing a dyld crash here. Try the following:

1) Enable SIP.
2) Build with a development credentials and move the binary into /Applications.
3) Attach log stream or the console to each of the processes launch the container app.

On macOS Big Sur Beta 10 this is how I tested with the new NETransparentProxyProvider API.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com