utun interfaces not cleaned up using NetworkExtension

Aloha. Opening and closing VPN tunnels results in as many utun interfaces as the amount of times the tunnel has been opened. These interfaces stay present and seem to be removed only upon system reboot.

We are using the NetworkExtension as a SystemExtension on macOS to create the virtual interfaces.

Is this the normal behaviour. Has anybody else experienced this?

utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1380
	inet6 fe80::8038:c353:17cd:c422%utun0 prefixlen 64 scopeid 0xf 
	nd6 options=201<PERFORMNUD,DAD>
utun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 2000
	inet6 fe80::cfb6:1324:d7e9:5d5%utun1 prefixlen 64 scopeid 0x10 
	nd6 options=201<PERFORMNUD,DAD>
utun2: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1300
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
utun3: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1300
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
utun4: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1300
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
utun5: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1300
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
utun6: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1300
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
utun7: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1300
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
utun8: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1300
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>

Is this the normal behaviour. Has anybody else experienced this?

I have experienced similar behavior, but not for all Network Extensions. Is this causing you issues or is your machine experiencing problems because of these interfaces?

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

We have clients complaining that this makes their machine become slow. It seems like the memory allocated in the PacketTunnelProvider is not released. Probably because the utun interface is not properly released.

If the process for the previous Network System Extension is still alive and has not been removed on the system then I could see where memory is being tied up here. However, just having an interface that used to be tied to a previously running packet tunnel would not still be consuming the memory from a process that no longer exists.

Having said that, if you are able to valid these performance concerns either with a sysdiagnose or otherwise then you should open a bug report for this. Please respond with the Feedback ID if you do.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

I am seeing this, and it seems that initial connections are slowed (may be DNS, may be the initial connection - I haven't had a chance to check that yet). What I do see though is this in the routing table:

Internet6: Destination Gateway Flags Netif Expire default fe80::%utun0 UGcI utun0
default fe80::%utun1 UGcI utun1
default fe80::%utun2 UGcI utun2
default fe80::%utun3 UGcI utun3
default fe80::%utun4 UGcI utun4
default fe80::%utun5 UGcI utun5

They do not appear in the IPv4 table at all, but they all seem to be listed as a default route (not even sure how multiple default routes is possible - seems counterintuitive). Oddly though, I cannot delete them from the route table either. When I try that I get this:

route delete -inet6 ::/0 fe80::%utun0

route: writing to routing socket: not in table delete net ::/0: gateway fe80::%utun0: not in table

I have the same issue, on macOS 13.4. I've implemented a sysExt custom VPN, and when connecting to the VPN, I see two new utuns. When disconnecting, only one of them is cleaned-up. While this weird behavior seems to work fine, things change when reaching a big number of utuns (around 95 utuns): Customers reported about slow connectivity, or sometimes - no connectivity at all. It seems that from that point (95 utuns) there are troubles with the DNS responses (No answers for DNS queries). Restarting the Mac will solve this problem, as it cleans the utuns.

As mentioned in this thread, the IPv6 routing table looks 'weird', but the IPv4 routing table looks good.

I have exit(0) at the code, after calling the stopTunnel completionHandler - so I tried to remove the exit(0), and tried to keep it but adding a 0.5 sleep before calling it - but it changed nothing.

  1. Is there any way to force clean those 'dirty' utuns?
  2. Any idea if a big num of utuns, can really cause the above (DNS/networking) problems?

It seems that from that point (95 utuns) there are troubles … Restarting the Mac will solve this problem, as it cleans the utuns.

That definitely sounds bugworthy. Please post your bug number, just for the record.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Update: I found the issue, which was related to my app, and not to the OS in any way.

@eskimo Thanks for the quick reply (as usual)! For some reason I didn't get a notification about it, so just saw it, but I've updated that this was a bug on my app.

I've updated that this was a bug on my app.

Interesting. Can you shared some details on how you managed to trigger this behaviour? I’m struggling to think of a way that your code could result in these symptoms.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@eskimo Sure, and actually, I'm still struggling with it. I'm connecting to the VPN from the containing app (startVPNTunnel). I assume this creates a utun. I'm also adding an observer for 'NEVPNStatusDidChange'. When I'm getting the status update for 'Connecting', I'm creating an IPC connection from the containing app to the system extension. For some reason, this creates a second utun. Disconnecting the VPN will clean only one of those utuns. So each connection/disconnection, results in an extra utun remaining on the system.

If I'm changing the logic a bit, and register to the IPC only when I'm getting the 'Connected' update (and not 'Connecting'), there will be only one utun, as expected. But I do need the IPC conenction ASAP, so I'm searching for a way to do it while 'Connecting', without creating another utun.

Addon: It seems that even calling to sendProviderMessage while at the 'connecting' phase will create an additional utun

Well, that’s weird. I got nothing. You might wait a day or two to see if Matt has any suggestions. If not, you have the usual options:

This is all production stuff, right? So on macOS 13, not macOS 14 beta?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

True - production, macOS 13. I'll do both options (bug + DTS) a bit later and will update here the relevant ticket. Thanks for the help!

Adding to the above: as an interim workaround: is there any way to get a positive, deterministic indication that a utun interface has been created by nesessionmanager BEFORE we try to send the IPC request to the extension? It seems that as "roee84" mentioned above, if the IPC registration is delayed a bit, until a utun is created, all is fine. If, on the other hand, we send an IPC registration request as soon as we recieve "Connecting" state, then it indeed results in two utuns being created (while only one is actually used).

Doesn't this increase the attack surface? It needs to be fixed because it is wasting resources and it's sloppy.When I do something like ifconfig, I get a huge list of over 100 utun interfaces left hanging. Obviously it's using memory and I expect impacts performance, at least of ifconfig if nothing else.

I;m using Global Protect VPN and it keeps incrementing the utun is uses.. I can clean up the last two, but can never get rid of the rest.. I've got utun0-6.

Macbook Pro running Sonomia 14.5

utun interfaces not cleaned up using NetworkExtension
 
 
Q