Why udp port 53 traffic is captured?

Is this a bug?

I would log out the flows that you are capturing on port 53 and then log out a small set of network rules for esting purpose; for example: 0.0.0.0 : 52 and 0.0.0.0 : 54 and if they overlap that you are still getting port 53 traffic, then yes, I would open a bug report. Please send me the Feedback ID so I can also take a look at your logs and extended examples here.



Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Rules with only

Code Block 
0.0.0.0 : 52 and 0.0.0.0 : 54

not going wrong, but it overlaps with rules like:

Code Block 
("0.0.0.0", "0...52") and ("0.0.0.0", "54...65535")

but it overlaps with rules like:
("0.0.0.0", "0...52") and ("0.0.0.0", "54...65535")

This is odd. What do your logs show if you log out your traffic being claimed when the overlapping takes place? In other words, how do you know it's overlapping and traffic for port 53 is entering handleNewFlow?


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

I have put in place some traced from within handleNewFlow together with the rule setting:
("0.0.0.0", "0...52") and ("0.0.0.0", "54...65535")
And I can get below traces

Code Block 
🤪FZ$--traffic handling udp flow to: 8.8.4.4:53	fzmacappproxy	04:16:22.468086+1000
🤪FZ$--traffic handling udp flow to: 1.1.1.1:53	fzmacappproxy	04:16:22.470011+1000
🤪FZ$--traffic handling udp flow to: 10.0.0.10:53	fzmacappproxy	04:16:22.471286+1000
🤪FZ$--traffic handling udp flow to: 10.0.0.10:53	fzmacappproxy	04:16:22.472517+1000
🤪FZ$--traffic handling udp flow to: 10.0.0.10:53	fzmacappproxy	04:16:22.473659+1000
🤪FZ$--traffic handling udp flow to: 10.0.0.10:53	fzmacappproxy	04:16:22.475028+1000

This is one thing.

The second thing is that with that rule set, sometimes it can capture all traffics, some times it seems blocked some where.
That can be verified by using netcat command sending out packets with the same subnet of en0.
If I filter with com.apple.nc on console app, with the command nc 192.168.0.100 8888, in good cases I can get below trace:

Code Block 
(2281480580): New flow: NEFlow type = stream, app = com.apple.nc, name = , 192.168.0.183:0 <-> 192.168.0.100:8888, filter_id = , interface = en0	fzmacappproxy	08:16:13.368577+1000
[Extension com.familyzone.macappproxy.proxy]: Calling handleNewFlow with TCP com.apple.nc[{length = 20, bytes = 0xb98252db9f8d8bbe4c383cf2ce6374eb34e3b83a}] remote: 192.168.0.100:8888 interface en0	fzmacappproxy	08:16:17.992887+1000
🤪FZ$--traffic handling tcp flow TCP com.apple.nc[{length = 20, bytes = 0xb98252db9f8d8bbe4c383cf2ce6374eb34e3b83a}] remote: 192.168.0.100:8888 interface en0	fzmacappproxy	08:16:17.992990+1000
[Extension com.familyzone.macappproxy.proxy]: provider accepted new flow TCP com.apple.nc[{length = 20, bytes = 0xb98252db9f8d8bbe4c383cf2ce6374eb34e3b83a}] remote: 192.168.0.100:8888 interface en0	fzmacappproxy	08:16:17.993701+1000

But sometimes it only show the first line below without the rest, which means NE callback is not envoked for some reason.

Code Block 
(95809956): New flow: NEFlow type = stream, app = com.apple.nc, name = , 192.168.0.183:0 <-> 192.168.0.100:8888, filter_id = , interface = en0	fzmacappproxy	08:19:05.315187+1000

Is it another bug?

🤪FZ$--traffic handling udp flow to: 8.8.4.4:53 fzmacappproxy 04:16:22.468086+1000
🤪FZ$--traffic handling udp flow to: 1.1.1.1:53 fzmacappproxy 04:16:22.470011+1000
🤪FZ$--traffic handling udp flow to: 10.0.0.10:53 fzmacappproxy 04:16:22.471286+1000

These flows looks like they are pointing to very specific DNS servers, are you also setting NEDNSSettings for domains? That would explain what you are seeing.

Regarding the second item:

But sometimes it only show the first line below without the rest, which means NE callback is not envoked for some reason.

Do you have a guard in the handleNewFlow method checking for a property of variable that is not set or nil?


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

There is a dns setting like below:

Code Block 
settings.dnsSettings = NEDNSSettings(servers: ["1.1.1.1", "1.0.0.1", "8.8.8.8", "8.8.4.4"])

Commenting it out still see the port 53 traffic like below:

Code Block 
🤪FZ$--traffic handling tcp flow TCP fc-system-service_darwin-amd64[{length = 20, bytes = 0xf2eb944db46831eb67c5114a4da698418011e2ad}] remote: 52.95.132.69:80 interface en0	fzmacappproxy	07:59:41.762307+1000
🤪FZ$--traffic handling udp flow to: 192.168.0.1:53	fzmacappproxy	07:59:41.989191+1000
🤪FZ$--traffic handling udp flow to: 192.168.0.1:53	fzmacappproxy	07:59:42.195246+1000
🤪FZ$--traffic handling udp flow to: 192.168.0.1:53	fzmacappproxy	07:59:42.196681+1000
🤪FZ$--traffic handling udp flow to: 192.168.0.1:53	fzmacappproxy	07:59:42.399203+1000
🤪FZ$--traffic handling udp flow to: fe80::1.53	fzmacappproxy	07:59:42.400754+1000
🤪FZ$--traffic handling udp flow to: fe80::1.53	fzmacappproxy	07:59:42.402079+1000
🤪FZ$--traffic handling udp flow to: 192.168.0.1:53	fzmacappproxy	07:59:42.609971+1000
🤪FZ$--traffic handling udp flow to: 192.168.0.1:53	fzmacappproxy	07:59:42.614274+1000
🤪FZ$--traffic handling udp flow to: 2001:4860:4860::8888.53	fzmacappproxy	07:59:53.037751+1000
🤪FZ$--traffic handling udp flow to: 10.0.0.10:53	fzmacappproxy	07:59:53.046197+1000
🤪FZ$--traffic handling udp flow to: 10.0.0.10:53	fzmacappproxy	07:59:53.048681+1000
🤪FZ$--traffic handling udp flow to: 1.1.1.1:53	fzmacappproxy	07:59:53.760442+1000
🤪FZ$--traffic handling udp flow to: 2001:4860:4860::8888.53	fzmacappproxy	07:59:55.095695+1000
🤪FZ$--traffic handling udp flow to: fe80::1.53	fzmacappproxy	07:59:56.042586+1000

handleNewFlow() is like below:

Code Block 
	override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool {
		if let TCPFlow = flow as? NEAppProxyTCPFlow {
			FZLog(.verbose, "traffic handling tcp flow \(TCPFlow.debugDescription)")
			TCPFlow.open(withLocalEndpoint: nil) { [unowned self] (error) in
				guard error == nil else {
					FZLog(.error, "tcp flow open error: \(error.debugDescription)")
					self.closeAppProxyFlow(flow: TCPFlow, error: error)
					return
				}
				self.handleFlow(tcpflow: TCPFlow)
			}
		} else if let UDPFlow = flow as? NEAppProxyUDPFlow {
			FZLog(.fatal, "UDP flow should not be captured \(UDPFlow.debugDescription) if handleNewUDPFlow is overridden")
		} else {
			FZLog(.fatal, "Should not capture something here \(flow.metaData.debugDescription)")
		}
		return true
	}

checking for a property of variable that is not set or nil?

What property of variable here do you mean? Code example?

The port 53 traffic is odd if you have excluded it from your NENetworkRule's and removed NEDNSSettings.

The extra inconsistent logging that you are also see if odd based on your code snippet.

If you are unable to resolve this, open a TSI and I, or Quinn, can take a deeper look at these two specific things.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

If I put in excluded rule with port 53, appproxy will keep recycling. Removing NEDNSSettings doesn't help with the two issues.

Raised a TSI case 767665663 only for udp 53 traffic issue. When I try to update with the other issue (handleNewFlow() sometimes not invoked) in the same scenario, couldn't find anywhere to edit my ticket. Do you know how to edit a TSI request?

Raised a TSI case 767665663 only for udp 53 traffic issue.

Thanks. I grabbed it and it's in my queue.

When I try to update with the other issue (handleNewFlow() sometimes not invoked) in the same scenario, couldn't find anywhere to edit my ticket. Do you know how to edit a TSI request?

Typically, if you respond on the TSI with more information, this should update the incident. However, I will remind you that a TSI is to cover one question, so if the port 53 traffic is the most important question, then let's focus on that.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Ok. Will raise a new one for it (handleNewFlow() sometimes not invoked). Seems need to pay then.

Are you using Catalina or Big Sur for this Network Extension?
I am using BigSur 11.2.3

So if you do not want port 53 UDP traffic this leaves you with a few options; first you could try to >filter only on a specific set of addresses or ranges that are not the default route (0.0.0.0). For >example, here is one that I used for all 17.0.0.0 traffic on port 443:
settings.includedNetworkRules = [
    NENetworkRule.init(remoteNetwork: NWHostEndpoint(hostname: "17.0.0.0", port: "443"),
                                 remotePrefix: 8,
                                 localNetwork: nil,
                                 localPrefix: 0,
                                 protocol: .any,
                                 direction: .outbound)
]
Next, you could proxy this traffic and handle it just like your regular UDP flows.

Thanks for suggesting touching the default route. That might be the way to go.
Can you say something more about usage of remotePrefix?
What I understand is value 8 means taking care of the first eight bits of the ip address which is 17 in this example. Any other examples showing different cases?

Also, I tried below setup, but still see udp port 53 traffic captured.
Thanks in advance for any suggestion.

Code Block 
		  var hosts = [("", "")]
		  var rules: [NENetworkRule] = []
			for seg in 1...254 {
				if seg != 127 {
						hosts = [("\(seg).0.0.0", "0...52"),
								 ("\(seg).0.0.0", "54...65535"),]
						rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 8, localNetwork: nil, localPrefix: 0, protocol: .UDP, direction: .outbound)
						rules.append(rule)
					}
				}
			}
      let settings = NETransparentProxyNetworkSettings.init(tunnelRemoteAddress: "127.0.0.1")
      settings.includedNetworkRules = rules
      settings.excludedNetworkRules = nil

Any other examples showing different cases?

Sure. It looks like you are adopting UDP based flows for 1...254.0.0.0/8 based traffic. I would recommend testing this and other TCP based NENetworkRule's out so that you are letting the system handle anything coming through on 0.0.0.0 and your proxy then presumably claiming all other traffic.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Following your suggestions, tested with below code capturing all port traffics:
Two things to mention:

1. It is better than before: network keeps alive for a couple of hours before it dies.

Attached the log when it crashed.
2. still seen udp 53 traffic.

Code Block 
			for seg in 1...254 {
				if seg != 127 {
					hosts = [("\(seg).0.0.0", "0")]
					for host in hosts {
						let ep = NWHostEndpoint(hostname: host.0, port: host.1)
						var rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 8, localNetwork: nil, localPrefix: 0, protocol: .TCP, direction: .outbound)
						rules.append(rule)
						hosts = [("\(seg).0.0.0", "0...52"),
								 ("\(seg).0.0.0", "54...65535"),]
						rule = NENetworkRule.init(remoteNetwork: ep, remotePrefix: 8, localNetwork: nil, localPrefix: 0, protocol: .UDP, direction: .outbound)
						rules.append(rule)
					}
				}
			}
			let settings = NETransparentProxyNetworkSettings.init(tunnelRemoteAddress: "127.0.0.1")
			settings.includedNetworkRules = rules
			settings.excludedNetworkRules = nil

Code Block 
[full mode crash](https://developer.apple.com/forums/content/attachment/88570304-6dc8-4bcb-a54e-5df782fdebca){: .log-attachment}

network keeps alive for a couple of hours before it dies.

I cannot see your crash log. You can attach it to the response or you can attach it to the existing TSI.

2. still seen udp 53 traffic.

If you are still seeing port 53 traffic come through then I suspect it is because of one of two reasons; first, because your NENetworkRule's localNetwork is nil and is matching UDP flows with an arbitrary port that falls in between 54...65535 on the local endpoint because NEAppProxyUDPFlow's only have a localEndpoint property. Then port 53 datagram traffic comes through when the local flow is opened. Second, that you are matching the remote network on outbound TCP traffic for port 53 traffic because you are using a wildcard for TCP traffic.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com