[NetworkExtension] Getting domain names from network flows' remoteEndpoints through a socket using NEFilterProvider?

Using NEFilterProvider, I'm able to look at the remoteEndpoint property for each network flow that comes from a socket. However, the hostname property is an IP address and not a URL/domain name, whereas doing the same for a flow that comes from a browser returns a URL. Is there a way to retrieve the domain name for a network flow that comes from the socket?
If you set the NWHostEndpoint on the NENetworkRules with hostnames do you see hostnames provided in the url property of NEFilterFlow in handleNewFlow?

A rough example of this might look like:
Code Block swift
let networkHostClear = NWHostEndpoint(hostname: "example.com", port: "80")
let networkHostSSL = NWHostEndpoint(hostname: "example.com", port: "443")
let clearNetworkRule = NENetworkRule(destinationHost: networkHostClear, protocol: .TCP)
let clearFilterRule = NEFilterRule(networkRule: clearNetworkRule, action: .filterData)
let sslNetworkRule = NENetworkRule(destinationHost: networkHostSSL, protocol: .TCP)
let sslFilterRule = NEFilterRule(networkRule: sslNetworkRule, action: .filterData)


Then in handleNewFlow:
Code Block swift
override func handleNewFlow(_ flow: NEFilterFlow) -> NEFilterNewFlowVerdict {
let url = flow.url?.absoluteString ?? "No URL"
os_log(.debug, log: self.log, "Flow URL %@", url)
}

Which produces the following logs when calling up example.com from Safari.
Code Block text
2020-07-07 07:34:17.464111-0700 0x34f1a5 Debug 0x0 : Flow URL http://example.com/
2020-07-07 07:34:39.839524-0700 0x34f1a5 Debug 0x0 : Flow URL https://example.com/


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Hi Matt,

I've tried this on both Monterey and Big Sur and it's working when navigating resources in Safari. However, the NEFilterFlow.url? is nil when using applications such as Firefox or Chrome. Also, most background network activity also does not have the url property filled out. (There are a few daemons like apsd and others for which this url field is filled—but most TCP/UDP traffic still has nil.)

I understand this may be a function of how DNS is resolved in client applications. Is there a more bulletproof, built-in way to correlate the resolved DNS names to the vast majority of the network traffic? I understand we can parse DNS ourselves and make these correlations, but I am initially looking for something like NEFilterFlow.url? but one that will not be nil most of the time.

Thank you

 Is there a more bulletproof, built-in way to correlate the resolved DNS names to the vast majority of the network traffic?

There is not. When working with other system traffic IPs, the hostnames will need to be derived by hand as you pointed out because only a small subset of the URLs on the system are available.

[NetworkExtension] Getting domain names from network flows' remoteEndpoints through a socket using NEFilterProvider?
 
 
Q