Lose Routing with NEPacketTunnelProvider

I have the code below to create an NEPacketTunnelProvider. I can inspect the packets and see the source and destination addresses and ports if the packets are tcp or udp etc. However, I lose all routing. My understanding is that if I write to the packetFlow the same data I read no routing would be lost. What am I doing wrong?



import NetworkExtension
import os.log

class AppProxyProvider: NEPacketTunnelProvider {
    
    func getIPAddress() -> String {
        var address: String?
        var ifaddr: UnsafeMutablePointer? = nil
        if getifaddrs(&ifaddr) == 0 {
            var ptr = ifaddr
            while ptr != nil {
                defer { ptr = ptr?.pointee.ifa_next }
                
                let interface = ptr?.pointee
                let addrFamily = interface?.ifa_addr.pointee.sa_family
                if addrFamily == UInt8(AF_INET) || addrFamily == UInt8(AF_INET6) {
                    
                    // wifi = ["en0"]
                    // wired = ["en2", "en3", "en4"]
                    // cellular = ["pdp_ip0","pdp_ip1","pdp_ip2","pdp_ip3"]
                    
                    let name: String = String(cString: (interface!.ifa_name))
                    if  name == "en0" || name == "en2" || name == "en3" || name == "en4" || name == "pdp_ip0" || name == "pdp_ip1" || name == "pdp_ip2" || name == "pdp_ip3" {
                        var hostname = [CChar](repeating: 0, count: Int(NI_MAXHOST))
                        getnameinfo(interface?.ifa_addr, socklen_t((interface?.ifa_addr.pointee.sa_len)!), &hostname, socklen_t(hostname.count), nil, socklen_t(0), NI_NUMERICHOST)
                        address = String(cString: hostname)
                    }
                }
            }
            freeifaddrs(ifaddr)
        }
        return address ?? ""
    }
  
    
    override func startTunnel(options: [String : NSObject]? = nil, completionHandler: @escaping (Error?) -> Void) {
        NSLog("Starting tunnel: '%@'", self.protocolConfiguration.serverAddress ?? "")
        
        let settings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: getIPAddress())
        
        settings.tunnelOverheadBytes = 81
        settings.mtu = 1200
        
        settings.ipv4Settings = NEIPv4Settings(addresses: [getIPAddress()], subnetMasks: ["255.255.255.255"])         settings.ipv4Settings?.includedRoutes = [NEIPv4Route.default()] // all routes
        settings.ipv4Settings?.excludedRoutes = [
            NEIPv4Route(destinationAddress: "0.0.0.0", subnetMask: "255.0.0.0"),
            NEIPv4Route(destinationAddress: "10.0.0.0", subnetMask: "255.0.0.0"),
            NEIPv4Route(destinationAddress: "127.0.0.0", subnetMask: "255.0.0.0"),
            NEIPv4Route(destinationAddress: "192.168.0.0", subnetMask: "255.255.0.0"),
            NEIPv4Route(destinationAddress: "255.255.255.255", subnetMask: "255.255.255.255"),
            NEIPv4Route(destinationAddress: "1.1.1.1", subnetMask: "255.255.255.255"),
            NEIPv4Route(destinationAddress: "1.0.0.1", subnetMask: "255.255.255.255"),
            NEIPv4Route(destinationAddress: "8.8.8.8", subnetMask: "255.255.255.255"),
            NEIPv4Route(destinationAddress: "8.8.4.4", subnetMask: "255.255.255.255")
        ] // avoid local routes and our dns servers
        // https://en.wikipedia.org/wiki/Reserved_IP_addresses

        //settings.dnsSettings = NEDNSSettings(servers: []) // just use the system dns servers
        // https://securitytrails.com/blog/dns-servers-privacy-security
        settings.dnsSettings = NEDNSSettings(servers: ["1.1.1.1", "1.0.0.1", "8.8.8.8", "8.8.4.4"])
        settings.dnsSettings?.matchDomains = [] // use system DNS
        //settings.dnsSettings?.matchDomains = [""] // override system dns
        
        self.reasserting = true
        
        self.setTunnelNetworkSettings(settings) { error in
            if let e = error {
                NSLog("Settings error %@", e.localizedDescription)
                completionHandler(e)
            } else {
                NSLog("Settings set without error")
                self.reasserting = false
                self.readPackets()
                
                completionHandler(nil)
            }
        }
    }
    
    override func stopTunnel(with reason: NEProviderStopReason, completionHandler: @escaping () -> Void) {
        NSLog("Stopping tunnel")
        completionHandler()
    }
    
    private func readPackets() {
        self.packetFlow.readPackets() { datas, protocolNumbers in
            
            self.sendPackets(datas, withProtocols: protocolNumbers)
            
            self.readPackets()
        }
    }
    
    private func sendPackets(_ data: [Data], withProtocols numbers: [NSNumber]) {
        self.packetFlow.writePackets(data, withProtocols: numbers)
    }
    
    private func processPacket(_ data: Data, protocolNumber: NSNumber) -> (Data, NSNumber) {
        let packet = IpPacket(withData: data, protocol: Protocol(rawValue: protocolNumber)!)
        NSLog("%@ packet created with %@ protocol src %@:%d dst %@:%d", packet.ipVersion.debugDescription, packet.transportProtocol.debugDescription, (packet.sourceAddress?.presentation)!, (packet.sourcePort ?? -1), (packet.destinationAddress?.presentation)!, (packet.destinationPort ?? -1))
        return (data, protocolNumber)
    }
 
}

Replies

I do not understand what “lose all routing” means. Please elaborate. Specifically, it would help if you explain your high-level goal. Are you actually trying to create a VPN? Or are you trying to use the VPN infrastructure for something else?

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thank you for the response eskimo!


I would just like to inspect the incoming packets.


By "lose all routing", I mean my connection no longer works - I can't browse anything and all internet functionality stops. I don't understand why since I am reading and then writing the exact same packets back out without modifciation. My understanding from the documentation is this should work.

I would just like to inspect the incoming packets.

Packet tunnel providers were created for VPN products and do not work well as a packet inspection mechanism. Your provider is responsible for passing all the packets it receives down your tunnel so that the VPN server can forward them appropriately.

I don't understand why since I am reading and then writing the exact same packets back out without modifciation.

You seems to have misunderstood how this is meant to work. You can’t ‘reject’ packets by echoing them back to the flow. Rather, a typical sequence works like this:

  1. An app opens a TCP connection to a server. This generates a TCP SYN packet.

  2. The kernel forwards that packet to your provider.

  3. Your provider reads it from the flow.

  4. It forwards it down the tunnel to the VPN server.

  5. The VPN server forwards it to the network (either a private network or, if you claim the default route, the Internet at large).

  6. The remote peer gets the TCP SYN and generates a SYN-ACK.

  7. This packet is routed to the VPN server.

  8. The VPN server forwards it down the tunnel to your provider.

  9. Your provider writes it to the flow.

  10. The kernel processes the packet and tells the app that the connection is in place.

  11. And so for the ACK that forms the last part of the three-way handshake, and all future traffic between the peers.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I believe I am running into a similar situation as below:

https://forums.developer.apple.com/thread/74464 which directly mentions my routing problem https://forums.developer.apple.com/thread/74194


> However, thinking about this further I suspect it won’t be possible because iOS does not give you access to raw IP or libpcap. I’m not sure though. Honestly, the mechanics of implement a NAT are outside of my area of expertise.


Also related https://forums.developer.apple.com/thread/76707https://forums.developer.apple.com/thread/36230Route raw IP packets to NEPacketTunnel's physical interface


I wonder how Surge or Charles is doing their captures - I would assume something similiar to NEKit