NEFilterDataProvider and NETransparentProxyProvider incompatiblity (breaks Safari websocket connections and other software)

Hello,

I guess we suffer from the same problem too (TSI 753464789). Apparently, that besides Safari websockets, it affects built-in ssh client if connected by hostname (broken pipe error) and any client program that uses NSStream/CFStream!
Console may give errors like "TestNSStream[1545:26660] SocketStream write error [0x10060b360]: 1 32"

Can you please confirm this with your tests and update your FB if appropriate?

In regards to the SSH case, excluding connections on port 22 for NEFilterDataProvider does workaround this issue. I realize that this may not work for every situation but this could be one possible workaround if you are facing a similar issue. The WebSocket issue is interesting because in my testing I have run many HTTP requests on port 443 and port 80 from Safari and been able to see these HTTP flows hit both NEFilterDataProvider and NEProxyTransparentManager. I have also been able to perform flow copying on the transparent proxy side without issue for Safari based flows between both providers. I have not tried WebSocket specific connections though via the TCP / HTTP upgrade process, so possibly this is the delta your are experiencing. My recommendation would be to open a TSI so I can dedicate some R&D time specifically for this scenario.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

I created TSI number 754458218 and specified more examples of apps that are broken by this scheme.

Thank you. I was able to find incident 754458218 and assigned it to my queue.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Just installed 11.1 RC and the bug was still not fixed. :(

So disappointing.

Will we need to keep our "Big Sur workarounds HOWTO" for at least one release?

Yes, this is disappointing.

Safari WebSocket and https://developer.apple.com/forums/thread/666894 (FB8925105) are affected by the same problem.

When the problem happens, Console outputs the following messages.

When using over HTTPS:
default 02:48:18.529584-0800 com.apple.WebKit.Networking CFNetwork SSLHandshake failed (-9810)
error 02:48:18.529628-0800 com.apple.WebKit.Networking TCP Conn 0x7fbabecf04e0 SSLHandshake failed (-9810)
default 02:48:18.531516-0800 com.apple.WebKit.Networking TCP Conn 0x7fbabecf04e0 canceled

But with HTTP we see the underlying problem:
error 02:46:23.243727-0800 com.apple.WebKit.Networking SocketStream write error [0x7fbabed57fc0]: 1 32
default 02:46:23.244424-0800 com.apple.WebKit.WebContent WebSocketChannel 0x10906b3d8 fail() reason='<private>'
default 02:46:23.245953-0800 com.apple.WebKit.Networking TCP Conn 0x7fbabed57fc0 canceled

Please notice "SocketStream write error [0x7fbabed57fc0]: 1 32". Error 32 is "broken pipe".

In my testing of this issue I was able to proxy the flows for the websocket connections successfully only if NEFilterDataProvider avoided the port. So, in this case the websocket flow would not hit the content filter first and be caught by the transparent proxy to handle the flow copying as normal. Obviously if your use case is to run a NEFilterDataProvider and a NEProxyTransparentProvider in tandem then this is a very slim workaround, if a workaround at all, but it should be noted. Also this is being addressed as part of (r. 71749910).

As for what is happening when the issue takes place, it looks like the TCP connection sets up successfully, but the websocket connection upgrade does not take place.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Thanks for the info. I hope it will be fixed soon.

Obviously if your use case is to run a NEFilterDataProvider and a NEProxyTransparentProvider in tandem then this is a very slim workaround,

I think that the tandem usage within the same app is a rare case.
What is much more frequent and confusing is a conflict between two apps from independent vendors.

Hello,

We have encountered this problem too (TSI 756181833). We have found multiple domains that cannot properly create WebSockets on Safari and we found that Microsoft Outlook fails to synchronize / connect the Exchange account. Safari displayed errors are exactly the same as above, while Microsoft Outlook logs indicated the following error "Microsoft Outlook: (CFNetwork) CFNetwork SSLHandshake failed (-9810)".

This happens while two separate network extensions, one implementing NETransparentProxyProvider and the other one implementing NEDataFilterProvider, NEDNSProxyProvider and NETransparentProxyProvider, run at the same time. We suspect that common includedNetworkRules, as other have described, seems to cause this behavior. Refusing to handle those problematic flows (bypassing the flow in handleNewFlow callback) does not help, it just passes the failure to the owner App, Safari or Microsoft Outlook in our case.

Hello! Are there any updates?

Hello! Are there any updates?

No. Please keep checking back on your bug report and checkout the newly released versions of macOS Big Sur for further updates here.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

We see the same behaviour with the Sophos Mac Endpoint product which uses the NETransparentProxyProvider subclass for its network extension and the CISC AnyConnect client which uses the NEFilterDataProvider.
When both are enabled and active, Safari will fail to connect to web.whatsapp.com and other sites that use websockets.
Google Chrome does not have a similar problem with the same configuration.

After updating to macOS 11.2 Beta 2 I was able to run a TCP filter with NEFilterDataProvider and a TCP NENetworkRule with NETransparentProxyProvider and make a wss:// connection successfully with websocket dot org. I was able to see the connection in Safari pass through the filter and then into the proxy.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com