Implement web filter in macOS with NetworkExtension API

How do you get flow as Browser Flow, something like NEFilterBrowserFlow but not just for WebKit-based browsers but for all browsers?

Yeah, NEFilterBrowserFlow is not available for macOS, but what you could do instead is attempt to filter on known ports that browser traffic uses, for example 80 and 443 and see where this gets you. I realize that you may miss some traffic here, but it may give you a baseline for comparison.

If #1 not possible, then how do you get something like sourceAppIdentifier to match against bundle identifiers of browser apps?

The sourceAppAuditToken may be available directly on the NEFilterFlow object delivered in handleNewFlow. It may also be worthwhile just taking a look at the the complete flow description to see what is all provided there when flows are delivered to your app.

If possible, how do you filter only HTTP traffic?

Check if you are getting a URL as well. let url = flow.url?.absoluteString will be provided is most cases and this should allow you to determine that they are HTTP flows. Whether they are browser based or not you will have to use sourceAppAuditToken to see if they are com.apple.safari based on not.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Thanks for your quick answer.

The sourceAppAuditToken may be available directly on the NEFilterFlow object delivered in handleNewFlow.

Yes, sourceAppAuditToken was suggested by Xcode's autocompletion feature. But, I'm unsure to extract the sourceAppIdentifier from it.
After searching through forums, I found following solution proposed by Quinn “The Eskimo!” in this thread.

Code Block swift
func bundleIDForAuditToken(_ tokenData: Data) -> String? {
    /* Get a code reference. */
    var codeQ: SecCode? = nil
    var err = SecCodeCopyGuestWithAttributes(nil, [
        kSecGuestAttributeAudit: tokenData
    ] as NSDictionary, [], &codeQ)
    guard err == errSecSuccess else {
        return nil
    }
    let code = codeQ!
    /* Convert that to a static code. */
    var staticCodeQ: SecStaticCode? = nil
    err = SecCodeCopyStaticCode(code, [], &staticCodeQ)
    guard err == errSecSuccess else {
        return nil
    }
    let staticCode = staticCodeQ!
    /* Get code signing information about that. */
    var infoQ: CFDictionary? = nil
    err = SecCodeCopySigningInformation(staticCode, [], &infoQ)
    guard err == errSecSuccess else {
        return nil
    }
    let info = infoQ! as! [String:Any]
    /* Extract the bundle ID from that. */
    guard
        let plist = info[kSecCodeInfoPList as String] as? [String:Any],
        let bundleID = plist[kCFBundleIdentifierKey as String] as? String
    else {
        return nil
    }
    return bundleID
}

However, when I tried, this function always returns nil.

It may also be worthwhile just taking a look at the the complete flow description to see what is all provided there when flows are delivered to your app.

On your suggestion, I inspected flow.description and yes it contains valuable information. Here's an example for sake of completion:

Code Block swift
"\n        identifier = CC71CAFA-781B-4402-970F-AB184EA5C14C\n        hostname = apidata.googleusercontent.com\n        sourceAppIdentifier = .com.apple.CalendarAgent\n        sourceAppVersion = \n        sourceAppUniqueIdentifier = 20:{length = 20, bytes = 0x618db8164f86a2db52185dd834f237c73551ca22}\n        procPID = 408\n        eprocPID = 408\n        direction = outbound\n        inBytes = 0\n        outBytes = 0\n        signature = 32:{length = 32, bytes = 0xb4f38a2e 3d2b14c3 24f807b6 06574c54 ... bbf1ebdb 08949f61 }\n        remoteEndpoint = 172.217.166.193:443\n        protocol = 6\n        family = 2\n        type = 1\n        procUUID = 1693DF56-EE62-3156-9B2D-23D06CF5A7A7\n        eprocUUID = 1693DF56-EE62-3156-9B2D-23D06CF5A7A7"

I've implemented following hacky solution to extract the sourceAppIdentifier from this string and returning early from non-browser app flows:

Code Block swift
class FilterDataProvider: NEFilterDataProvider {
    static let filteredApps: Set = [".com.apple.Safari", ".com.google.Chrome", ".org.mozilla.firefox"]
    override func startFilter(completionHandler: @escaping (Error?) -> Void) {
        /* Same as above */
    }
    func isFiltered(description: String) -> Bool {
        guard let sourceAppIdentifier = description.matches(for: #"(?<=sourceAppIdentifier = ).+?(?=\s)"#).first else {
            return false
        }
        return FilterDataProvider.filteredApps.contains(sourceAppIdentifier)
    }
    override func handleNewFlow(_ flow: NEFilterFlow) -> NEFilterNewFlowVerdict {
        guard let url = flow.url?.absoluteURL,
              isFiltered(description: flow.description) else {
                return .allow()
        }
				/* .drop() if url found in local DB */
        return .allow()
    }
}
extension String {
    func matches(for regex: String) -> [String] {
        do {
            let regex = try NSRegularExpression(pattern: regex)
            let results = regex.matches(in: self, range: NSRange(self.startIndex..., in: self))
            return results.map {
                String(self[Range($0.range, in: self)!])
            }
        } catch {
            return []
        }
    }
}

Is there an alternative to Quinn's solution or any better way to do this?

While testing, I also noticed that flow from browsers other than Safari does not provide url and hence is of no use. In handleNewFlow the flow.url is nil for Chrome and Firefox.

Documentation describes this:

This parameter is only non-nil for flows that originate from WebKit browser objects.

Any possible solution for getting url on flows from non-WebKit browsers?

Is there an alternative to Quinn's solution or any better way to do this?

Quinn's approach is still the recommended way here.

You could try adding kSecCodeInfoIdentifier to see i this produces anything more?

Code Block swift
let info = infoQ! as! [String:Any]
/* If the info identifier is available, return this. */
if let identifier = info[kSecCodeInfoIdentifier as String] as? String {
    return identifier
}
/* Othewise look for the identifier in the plist */
guard
    let plist = info[kSecCodeInfoPList as String] as? [String:Any],
    let bundleID = plist[kCFBundleIdentifierKey as String] as? String
else {
    return nil
}

However, if you essentially get the infoQ! as! [String:Any] dictionary from SecCodeCopySigningInformation here, take a look at all of the keys available. One of them may provide more information in a different context than others.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Hello
I have been trying to do similar thing - filter outbound HTTP requests from Safari by URL. I am also using SimpleFirewall app provided as a sample for  WWDC19 session about Network Extension. But when I try to get url of the flow originated from Safari always getting nil of string with address. I have modified only

Code Block 
handleNewFlow

And here it is:

Code Block swift
    override func handleNewFlow(_ flow: NEFilterFlow) -> NEFilterNewFlowVerdict {
        guard let socketFlow = flow as? NEFilterSocketFlow,
            let remoteEndpoint = socketFlow.remoteEndpoint as? NWHostEndpoint,
            let localEndpoint = socketFlow.localEndpoint as? NWHostEndpoint else {
                return .allow()
            }
        let blacklisted = "www.example.com"
        os_log("Flow %@", flow)
        if socketFlow.direction != .outbound
        {
            os_log("Allow non-outbound connections")
            return .allow()
        }
        if  socketFlow.url != nil
        {
            os_log("Host %@", socketFlow.url?.host as! CVarArg)
            if blacklisted == socketFlow.url?.host
            {
                return .drop()
            }
        }
        else
        {
            os_log("URL is nil")
        }
        os_log("Endpoint hostname %@", remoteEndpoint.hostname)
        os_log("New flow with local endpoint %@, remote endpoint %@", localEndpoint, remoteEndpoint)
        let flowInfo = [
            FlowInfoKey.localPort.rawValue: localEndpoint.port,
            FlowInfoKey.remoteAddress.rawValue: remoteEndpoint.hostname
        ]
        let prompted = IPCConnection.shared.promptUser(aboutFlow: flowInfo) { allow in
            let userVerdict: NEFilterNewFlowVerdict = allow ? .allow() : .drop()
            self.resumeFlow(flow, with: userVerdict)
        }
        guard prompted else {
            return .allow()
        }
        return .pause()
    }

When I run a firewall and check log in console I always see "URL is nil" message when I try to open websites from Safari
Could you help to understand what am I doing wrong?

@yblk

I suspect that you are seeing a lot of flows just being allowed. Try removing this in your guard:

Code Block swift
let localEndpoint = socketFlow.localEndpoint as? NWHostEndpoint 

A lot of flows that pass through here do not have a localEndpoint yet and so this is nil. Also, some flows that pass through here will just have nil set for flow.url?.absoluteString, so you may need to drop back to examining traffic for app or IP, as discussed on this thread.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Hi @Systems Engineer @rmalviya @yblk , Matt

DESCRIPTION OF PROBLEM :- We are currently developing a macOS app using the NEFilterDataProvider in the Network Extension framework, and we've encountered an issue regarding hostname resolution that we would like your guidance on.

In our implementation, we need to drop network flows based on the hostname. The app successfully receives the remoteHostname or remoteEndpoint.hostname for browsers such as Safari and Mozilla Firefox. However, for other browsers like Chrome, Opera Mini, Arc, Brave, and Edge, we only receive the IP address instead of the hostname.

We are particularly looking for a way to retrieve the hostname for all browsers to apply our filtering logic consistently. Could you please advise whether there is any additional configuration or API we can use to ensure that we receive hostnames for these browsers as well? Alternatively, is this a limitation of the browsers themselves, and should we expect to only receive IP addresses for certain cases?

STEPS TO REPRODUCE :- For Chrome, Brave, Edge, and Arc browsers you won't receive the hostname in NEFilterFlow.

Using the same sample project provided in WWDC 2019 https://developer.apple.com/documentation/networkextension/filtering_network_traffic

SUPPORT INFORMATION :- Did someone from Apple ask you to submit a code-level support request? No

Do you have a focused test project that demonstrates your issue? Yes, I have a focused test project to submit with my request

What code level support issue are you having? Problems with an Apple framework API in my app