There is no way to do that, but it seems like a perfectly reasonable enhancement request.
Please post your bug number, just for the record.
Thanks for the suggestion! I've filed FB11655901.
Post
Replies
Boosts
Views
Activity
Hi!
Just wanted to mention that I've also bumped into this exact issue (duplicate queries to different DNS servers), although I'm NOT trying to filter all DNS traffic: I was able to reproduce DNS queries "leaking" past my tunnel even when matchDomains in my NEDNSSettings contains just a few specific domains that I want to be resolved through the tunnel.
In my case, this seems to only reproduce in third-party apps, Safari for example doesn't exhibit this behaviour. I've filed a bugreport: FB9126588 and attached a simple reproducer app and tunnel to it.
Yep, tried adding rules for either/both 127.0.0.1/32 and ::1/128, with specific ports and zero port: no luck.
I'm seeing something like this in the Console:
[Extension com.i.love.apple.APIs.NEService]: provider set tunnel configuration to
tunnelRemoteAddress = 9-char-str
includedNetworkRules = (
{
matchRemoteEndpoint = ::1.0
matchRemotePrefix = 128
matchProtocol = 3-char-str
matchDirection = 8-char-str
appliesToLoopback = NO
},
{
matchRemoteEndpoint = ::1.443
matchRemotePrefix = 128
matchProtocol = 3-char-str
matchDirection = 8-char-str
appliesToLoopback = NO
},
{
matchRemoteEndpoint = 127.0.0.1:0
matchRemotePrefix = 32
matchProtocol = 3-char-str
matchDirection = 8-char-str
appliesToLoopback = NO
},
{
matchRemoteEndpoint = 127.0.0.1:443
matchRemotePrefix = 32
matchProtocol = 3-char-str
matchDirection = 8-char-str
appliesToLoopback = NO
},
)
...
I think it might have something to do with appliesToLoopback = NO, though I'm not quite sure...
@dverevkin Hi! Could you please submit your patch or open a bug report with what you could find out to Dart? I've looked through their GitHub issues and got the impression that the Dart developers don't have a clue what's happening or how to fix it, while a few people are having this exact issue in different circumstances.
Hi, @meaton!
Is there any news regarding this issue? I can still reproduce on macOS 11.2 (20D64).
Hi, @meaton
I stand corrected; Initially I was under the impression that any kind of network extension is affected, but after careful checking, it's only NETransparentProxyProvider that is causing trouble.
The included sample always returns NO from handleNewFlow::
(BOOL)handleNewFlow:(NEAppProxyFlow *)flow {
		return NO;
		// commented code omitted
}
, so it's safe to assume that any attempt to handle this flow is actually the incorrect behaviour of the API.
This also means that only macOS 11 is affected (unless NETransparentProxyProvider has been ported to Catalina as well).
Hi, @meaton!
Makes sense, but can I still filter them somehow?
The use case is filtering traffic that goes between a browser and a local proxy like ShadowSocks or Tor. Note that filtering the outbound connection of the local proxy itself is problematic because it is encrypted (with a custom protocol) so I won't be able to do anything with it.
Hi, @meaton! It seems, on the new macOS 10.0.1 beta, with NETransparentProxyProvider, issues reported in this thread, including the proxy issue, are resolved. Thanks for the support!
@meaton Regarding proxy issues: I've tried your suggestion to set the proxySettings property of NETransparentProxyNetworkSettings. I've tried something like this (just to test if it will help)
NETransparentProxyNetworkSettings *sets = [[NETransparentProxyNetworkSettings alloc] initWithTunnelRemoteAddress:@"127.0.0.1"];
sets.proxySettings = [[NEProxySettings alloc] init];
sets.proxySettings.HTTPEnabled = YES;
sets.proxySettings.HTTPServer = [[NEProxyServer alloc] initWithAddress: @"192.168.1.11" port:12345];
sets.proxySettings.HTTPSEnabled = YES;
sets.proxySettings.HTTPSServer = [[NEProxyServer alloc] initWithAddress: @"192.168.1.11" port:12345];
The behaviour is identical to when the proxy settings are just set through the system settings: my app receives a flow with remoteEndpoint set to <website hostname>:<proxy port>, like this:
TCP com.apple.CalendarAgent[{length = 20, bytes = 0xefd63d533d62ec86c8d6149c2bf2699b909a37ac}] remote: p45-caldav.icloud.com:12345
I don't see how I can make a connection when I receive a wrong hostname:port pair. I might have misunderstood your suggested solution, would you please elaborate?
@meaton, Hi! I'm trying to run my app after changing the base class of my provider from NEAppProxyProvider to NETransparentProxyProvider and I'm getting a SIGCONT here:
Thread 1#0 0x0000000118c23eb4 in ImageLoaderMachOCompressed::updateOptimizedLazyPointers(ImageLoader::LinkContext const&) ()
#1 0x0000000118c22720 in ImageLoaderMachOCompressed::doBind(ImageLoader::LinkContext const&, bool, ImageLoader const*) ()
#2 0x0000000118c15b71 in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#3 0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#4 0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#5 0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#6 0x0000000118c15b3a in ImageLoader::recursiveBind(ImageLoader::LinkContext const&, bool, bool, ImageLoader const*) ()
#7 0x0000000118c07955 in dyld::_main(macho_header const*, unsigned long, int, char const, char const, char const**, unsigned long*) ()
#8 0x0000000118c0022b in dyldbootstrap::start(dyld3::MachOLoaded const*, int, char const**, dyld3::MachOLoaded const*, unsigned long*) ()
#9 0x0000000118c00025 in _dyld_start ()
After typing continue in the debugger, the program crashes. Can you please help me with this? The documentation for NETransparentProxyProvider is currently blank.
I'm using Xcode Version 12.2 beta 3 (12B5035g), should I be using something else?
Hmm, it seems that now even after going back to NEAppProxyProvider the app won't start.
Maybe it's not related to the change, and instead an issue with the latest xcode beta.
Hi! I've retried on Big Sur beta 10 and everything still reproduces, unfortunately (all remote hosts seen as 0.0.0.0 in Safari and SSH is particularly depressing)
I've also encountered a new problem, should I open a separate bug report? The problem is as follows: Configure a HTTP and HTTPS proxy in system settings for your network interface. The proxy must be in your local network, NOT on a localhost.
Now start a transparent proxy extension.
Open any site in Safari.
Transparent proxy receives a flow with the site's domain name and proxy's port as the remote endpoint.
Obviously, traffic is not routed through system-configured proxy, and the port is wrong, so Internet completely breaks :(
I'll attach some clarifying screenshots to the feedback.
By the way, I've updated the feedback with all the requested information and all the additional issues that I've encountered so far with hopefully more clear steps to reproduce, and even a screen cast. Please take a look.
Thanks for looking into this!
I've updated the feedback with a revised sample that opens the flow after outbound connection has been established, and sets the local endpoint accordingly. Unfortunately, this doesn't seem to help. I've also reproduced an issue with Safari seeing 0.0.0.0 and how it breaks Youtube, and attached some screenshots.
Please note that disabling IPv6 is crucial to reproducing this bug, and also the issue itself is not with receiving emails, but just with trying to add a Yahoo account in Mail. And, of course, there's also the newly uncovered issue with attachments not working in Feedback Assistant, when the same network extension is active.
By the way, I was under the impression that such broad rules as in your example are prohibited? I'm referring to these statements from the documentation (https://developer.apple.com/documentation/networkextension/netransparentproxynetworksettings/3143656-includednetworkrules?language=objc):
If the port string of the endpoint is 0 or is the empty string, then the address of the endpoint must be a non-wildcard address, such as 0.0.0.0 or ::. If the address is a wildcard address (such as 0.0.0.0 or ::), then the port string of the endpoint must be non-empty and must not be 0.
It turns out, attachments in Feedback Assistant are also affected by the presence of a transparent proxy network extension.
I've updated the feedback once more, now my data seems to be successfully attached.
It looks like for IPv6 there's already a workaround in place:
➜ ~ /usr/bin/ssh fa2.lamo.su
The authenticity of host 'fa2.lamo.su (0.0.0.1)' can't be established.
ECDSA key fingerprint is SHA256:dZsSQrUwwPgm/EZ8wGkuijpIEslPTg3NcfIEQgWLqiY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? no
Host key verification failed.
➜ ~ /usr/local/Cellar/openssh/8.3p1/bin/ssh fa2.lamo.su
The authenticity of host 'fa2.lamo.su (2a01:4f8:1c17:4e80::1)' can't be established.
ECDSA key fingerprint is SHA256:dZsSQrUwwPgm/EZ8wGkuijpIEslPTg3NcfIEQgWLqiY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? no
Host key verification failed.
Note how Apple's SSH sees some probably synthesized IPv4 address when trying to connect to IPv6 host through a Transparent Proxy. To reproduce this, I've enabled IPv6 on my machine and added 2000::/3 to includedNetworkRules in my NETransparentProxyNetworkSettings
I'm also seeing synthesized IPv4 addresses in Safari inspector:
Summary
URL: https://www.youtube.com/
Status: 200
Source: Network
Address: 0.0.0.198:443
Summary
URL: https://www.google.com/
Status: 200
Source: Network
Address: 0.0.0.106:443
I can't see this "synthetic" IPv4 connection in netstat
Please note that if there's no IPv6 connectivity on user's machine, we go back to 0.0.0.0 issue.