Hundreds of appproxy flows are generated when visiting some web site.

Hi there,

During the test when using appproxy from systemextension (TransparentProxyProvider) capturing 80/443 tcp traffic, it is found that while visiting some website, there are hundreds of tcp flows generated, causing the appproxy stuck for some period (if we have some tasks for each flow). The websites can be some speedtest sites, such as https://www.speedtest.net or some complex forums, for example, https://www.wenxuecity.com

Do you have any suggestion on how to reduce the number of flows?

Thanks in advance.

Regards Richard

Can you post your NENetworkRules that are being used when you test this?

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Thanks Matt for the reply. Paste below the rules setting.

	private func buildRules(fullMode: Bool) -> [NENetworkRule] {
		var hosts = [("", "")]
		var rules: [NENetworkRule] = []

	    hosts = [("0.0.0.0", "80"),
					("0.0.0.0", "443")]
		for host in hosts {
			let ep = NWHostEndpoint(hostname: host.0, port: host.1)
			let rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol: .TCP, direction: .outbound)
			rules.append(rule)
		}

		for seg in 1...223 {
			if seg != 127 {
				let ep = NWHostEndpoint(hostname: "\(seg).0.0.0", port: "0")
				// capture all udp traffic including port 53
				var rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 8, localNetwork: nil, localPrefix: 0, protocol: .UDP, direction: .outbound)
				rules.append(rule)
			}
		}

		return rules
	}

	private func exceptRules() -> [NENetworkRule] {
		var hosts = [("", "")]
		var rules: [NENetworkRule] = []

		// rule out dhcp, ntp traffic
		hosts = [("0.0.0.0", "67"),
				 ("0.0.0.0", "68"),
				 ("0.0.0.0", "123")]
		for host in hosts {
			let ep = NWHostEndpoint(hostname: host.0, port: host.1)
			let rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol: .UDP, direction: .outbound)
			rules.append(rule)
		}

		return rules
	}

	override func startProxy(...) {
        ...
		let settings = NETransparentProxyNetworkSettings.init(tunnelRemoteAddress: "127.0.0.1")
		settings.includedNetworkRules = buildRules()
		settings.excludedNetworkRules = exceptRules()
		setTunnelNetworkSettings(settings) {
        ...
        }
    }

Attached also traces with these two websites. Only printout target url when a difference flowId (flows.hash) request is up. We can see that:

  1. The number of request is significant
  2. Even with different flowId, there are many continuous request with the same url. Not sure how it is decided to generate a new flowId.

Thanks in advance for the support.

Richard

Attach another log with flowId (tcpflow.hash) on display. You can see that when visiting speedtest.net, although some traffic with the same url, their flowId are different. Is it possible to have only one flowId generated with the same url (tcp)?

[speedtest.net.txt](https://developer.apple.com/forums/content/attachment/7ab24bbc-507e-40f9-a256-e160e0727131)

https://fastlane.rubiconproject.com, flowId: 416043267
https://fastlane.rubiconproject.com, flowId: 3175036210
https://fastlane.rubiconproject.com, flowId: 1261766144

And the question also is: with a new flowId request in, we will create a new tcp connection (so that is quite a lot!). Is that correct?

And is tcpflow.hash alway unique? Means if I use that value to distinguish different tcpflow, will corrupt happens?

And the question also is: with a new flowId request in, we will create a new tcp connection (so that is quite a lot!). Is that correct?

I took a look at the specific case with https://www.speedtest.net and you are correct, it does appear that these client web sites are opening many connections to run a performance test. I seen around 160 TCP and UDP flows on my proxy's end. You can also see this in the Safari Web Inspector if you look at the Network tab. So, each flow that is generated by a client website, that is also claimed by the proxy, will have a specific hash to it that is unique because it conforms to the NSObjectProtocol.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

So, each flow that is generated by a client website, that is also claimed by the proxy, will have a specific hash to it that is unique because it conforms to the NSObjectProtocol.

Thanks Matt for the reply.

Can you help confirm that each flow coming in has an unique flow.hash value by refreshing the same website?

I put a trace inside override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool and printout flow.hash, and it seems unique each print. This is under BigSur 11.4.

But I do remember the same trace under Catalina had some flow.hash printed out of the same value. Not sure is it something changed?

Thanks in advance for any suggestion.

Richard

Can you help confirm that each flow coming in has an unique flow.hash value by refreshing the same website?

This is something that you should be able to confirm using the techniques that you have mentioned. If you find that you have found a bug here, please open a bug report.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Hundreds of appproxy flows are generated when visiting some web site.
 
 
Q