But with that configuration, I still set traffic to apple.com 443 as mentioned: 🤪FZ$--hostName: www.apple.com remoteEp: 23.202.173.153:443 appId: com.google.Chrome.helper fzmacappproxy 18:22:57.007704+1000 🤪FZ$--hostName: supportmetrics.apple.com remoteEp: 17.137.160.100:443 appId: com.google.Chrome.helper fzmacappproxy 18:23:47.128264+1000 🤪FZ$--hostName: developer.apple.com remoteEp: 17.253.67.203:443 appId: com.google.Chrome.helper fzmacappproxy 18:23:47.485234+1000

Code Block swift let settings = NETransparentProxyNetworkSettings(tunnelRemoteAddress: "x.x.x.x") settings.includedNetworkRules = [ NENetworkRule(remoteNetwork: NWHostEndpoint(hostname: "0.0.0.0", port: "443"), remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol:.TCP, direction: .outbound) ] settings.excludedNetworkRules = [ NENetworkRule(remoteNetwork: NWHostEndpoint(hostname: "apple.com", port: "0"), remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol:.TCP, direction: .outbound) ] And see all *.apple.com traffic go direct as normal, while all port 443 traffic outside of .apple.com went to the proxy. That means no difference between with and without that exclude rule :) Is that right?

Thanks for sharing. Do you have any update on the status of r. 70246597, r. 66312716 or r. 67923914? I cannot find any result form https://openradar.appspot.com/ When (beginning from which macOS version) can this uninstall sysExt issue be resolve? I tried Monterey, and it is of the same issue. I see questions like this a lot so I want to clarify one thing: The SystemExtensions framework is intended to be used by a GUI app to load and unload system extensions in response to user actions. It’s not meant to be called from a command-line tool, regardless of how that command-line tool is packaged. Calling it from a command-line tool is unsupported and may yield weird results. I recently filed a bug to get this clarified in the framework’s documentation (r. 70246597). We also have a bug on file to add more options for installing and uninstalling system extensions (r. 66312716), although if you have specific requirements in that space it wouldn’t hurt to file your own bug describing those requirements. Finally, a lot of folks who ask about this are primarily focused on deploying to managed environments, and such environments can do more with system extensions using MDM. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@apple.com" Uninstall is a major hole in the sysex story right now )-: For example, you can currently install a sysex on a managed Mac but not uninstall it. The sysex team is aware of these problems (r. 67923914) but I’ve no info to share as to when they might be resolved. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Multiline BlockQuote

flowHash.txt So, each flow that is generated by a client website, that is also claimed by the proxy, will have a specific hash to it that is unique because it conforms to the NSObjectProtocol. Thanks Matt for the reply. Can you help confirm that each flow coming in has an unique flow.hash value by refreshing the same website? I put a trace inside override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool and printout flow.hash, and it seems unique each print. This is under BigSur 11.4. But I do remember the same trace under Catalina had some flow.hash printed out of the same value. Not sure is it something changed? Thanks in advance for any suggestion. Richard

And is tcpflow.hash alway unique? Means if I use that value to distinguish different tcpflow, will corrupt happens?

And the question also is: with a new flowId request in, we will create a new tcp connection (so that is quite a lot!). Is that correct?

Attach another log with flowId (tcpflow.hash) on display. You can see that when visiting speedtest.net, although some traffic with the same url, their flowId are different. Is it possible to have only one flowId generated with the same url (tcp)? [speedtest.net.txt](https://developer.apple.com/forums/content/attachment/7ab24bbc-507e-40f9-a256-e160e0727131) https://fastlane.rubiconproject.com, flowId: 416043267 https://fastlane.rubiconproject.com, flowId: 3175036210 https://fastlane.rubiconproject.com, flowId: 1261766144

Thanks Matt for the reply. Paste below the rules setting. speedtest.net.txt wenxuecity.com.txt private func buildRules(fullMode: Bool) -> [NENetworkRule] { var hosts = [("", "")] var rules: [NENetworkRule] = [] hosts = [("0.0.0.0", "80"), ("0.0.0.0", "443")] for host in hosts { let ep = NWHostEndpoint(hostname: host.0, port: host.1) let rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol: .TCP, direction: .outbound) rules.append(rule) } for seg in 1...223 { if seg != 127 { let ep = NWHostEndpoint(hostname: "\(seg).0.0.0", port: "0") // capture all udp traffic including port 53 var rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 8, localNetwork: nil, localPrefix: 0, protocol: .UDP, direction: .outbound) rules.append(rule) } } return rules } private func exceptRules() -> [NENetworkRule] { var hosts = [("", "")] var rules: [NENetworkRule] = [] // rule out dhcp, ntp traffic hosts = [("0.0.0.0", "67"), ("0.0.0.0", "68"), ("0.0.0.0", "123")] for host in hosts { let ep = NWHostEndpoint(hostname: host.0, port: host.1) let rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol: .UDP, direction: .outbound) rules.append(rule) } return rules } override func startProxy(...) { ... let settings = NETransparentProxyNetworkSettings.init(tunnelRemoteAddress: "127.0.0.1") settings.includedNetworkRules = buildRules() settings.excludedNetworkRules = exceptRules() setTunnelNetworkSettings(settings) { ... } } Attached also traces with these two websites. Only printout target url when a difference flowId (flows.hash) request is up. We can see that: The number of request is significant Even with different flowId, there are many continuous request with the same url. Not sure how it is decided to generate a new flowId. Thanks in advance for the support. Richard

Thanks a lot Matt for the. reply. That's clear.

And filter out only tcp 80 and 443 as I do.

Hi Matt, Confirmed that beginning from BigSur 11.4 the network loss issue happens: I tried on two macbook pro. They have no problem when at BigSur 11.3.1. But as soon as I upgrade them to 11.4, issue happens. Already use NETransparentProxy. Can you help double confirm? You can use fast.com and speedtest.net which can show the symptom rather quick. Thanks in advance.

Hi Matt, About bringing up both dns and transparent proxy network devices, can you share with me your code on how to start them only after the other? I tried different combinations inside AppDelegate.swift, with below way, only sometimes I can get both network devices added, other times only appproxy device can be seen. func applicationDidFinishLaunching(_ aNotification: Notification) {     ProxyManager.shared().loadProxyManager(connect: true) {       ProxyManager.shared().addObserver {}       ProxyManager.shared().start()       DNSManager.shared().start() {}     }   } About stability thing with both proxy running, have you observe its stability over time? I tried with below script and found sometimes it just get stuck there for a while then resume going on. #!/bin/bash i=0 while true do   i=$((i+1))   echo "loop $i"   nslookup www.google.com   sleep 1 done

Using NEAppProxyProvider or even better yet, NETransparentProxyProvider is one known alternative to creating a NEDNSProxyProvider to handle your system level DNS flows. Good to see that confirmation. As far as your name resolution issues, if you start your NEDNSProxyProvider to handle DNS flows and then you start a NEAppProxyProvider or NETransparentProxyProvider afterwards to only handle TCP flows, do you still run into a conflict here? Tried before as remember with out udp filtering from appproxy with the same problem (network cannot pass through). Can double confirm anyway. Is there any concern who to start first, dnsproxy or appproxy?

If I am reading this correctly, NWConnection is not broken in macOS 11.5 Beta but you are instead seeing variable network speeds through your provider where NWConnection is used. Is that correct? It is not only network speed drop, but after some time, no network access anymore (I am capturing tcp 80/443 outbound traffic only). That was not the case with bigSur 11.4. Are you using NEAppProxyProvider or are you using NETransparentProxyProvider? I am using NEAppProxyProvider.

Hi Matt, We still cannot make AppProxyProvider and DNSProxyProvider working together. Main problems still are: Stability thing  After running for a while with both proxies, name resolution failed. Need uninstall extension each time before you run the application. It is with below command to control the extension. systemextensionsctl uninstall 5S77G864UH com.familyzone.macappproxy.dns However, we made it work by using only AppProxyProvider to filter and handle dns traffics based on udp port 53. The problem is that is this way illegal, as nowhere I can find in the document saying that is a valid way of handling dns traffic using only AppProxyProvider. So, need your help in two respects: Solve the issues having both AppProxyProvider and DNSProxyProvider working together. Help confirm whether it is legal to handle dns traffic with AppProxyProvider only? Thanks in advance. Richard