The routing table's behavior got weird after Network Extension runs in System Extension

Hello, everyone ~~

I am developing an App with Network Extension which runs in App Extension. When execute the app and it made a tunnel connection to the server. Routing Table is correct. Send a PING to my server through the tunnel, It works fine.

After then, I try to distribute (Notarized) my APP with "Developer ID" and TEST it on my Dev iMac, But It didn't work. (In this time, I didn't move the Network Extension from App Extension to System Extension)

Because, The developer Try to distribute an app offline who has to move the Network Extension from App Extension to System Extension.
Finally I got the right CODESIGN to the APP and notarized by Apple.
Code Block
thank you eskimo

My App is working, I got my System Extension which appears in the list who was typing the command as below in the "Terminal.app"
Code Block
$ systemextensionctrl list

And I saw the System Extension process also appears in "Activity Monitor.app"

After then, I send a PING to my server. It seems to the server got the ICMP request packets and response to the App. At App side(Client Side), real NIC interface (eth0) seems to got the packets but those packets was not read by my program which is running inside Network Extension.
And uTun virtual interface also are not seems to got any packets. I watched those behaviors by "Wireshark".

I assumed that may be CODESIGN caused those problems. So, I unchecked those options in "Hardened Runtime" as below. But It doesn't works.
  1. Allow Unsigned Executable Memory

  2. Disable Library Validation

  3. Disable Executable Memory protection

Has anyone succeed to distribute the app with Network Extension offline? And It works normally?

What's problem with Routing Table? (May be Routing Table is not the suspect)

If no one has been distributed the app with Network Extension which runs in System Extension. It's a kind of disaster to me.

If so, I have to report this issue to my boss with an evidence why It's impossible to distribute offline.
And then I have to change entire program process without Network Extension.

If someone knows the reason, please tell me.



Help!!!!

I assumed that may be CODESIGN caused those problems

ICMP packets not going through your tun interface does not sound like an issue related to codesign or notarization. What happens if you build your Network Extension locally on a development machine and test this functionality?

Also, can you test this by binding ping to a specific interface, in this case your tun interface, and see if that provides you anything? For example, take a look at man ping and you should see an Apple specific option for boundif. Try forcing this to your tun interface. % ping -b tun2 apple.com


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Yes, You are right. It's my bad.

The suspect was not the codesign.

I am using
Code Block
socketpair()
to monitoring what's going on in uTun .
The code as below.
Code Block
CFSocketContext socketCtx = {0, (__bridge void *)context, NULL, NULL, NULL};
   
tunnelSideSock = CFSocketCreateWithNative(kCFAllocatorDefault, socks[UTUN_SIDE], kCFSocketDataCallBack|kCFSocketReadCallBack|kCFSocketWriteCallBack, callback, &socketCtx);
   
  CFRunLoopSourceRef tunSocketSource = CFSocketCreateRunLoopSource(kCFAllocatorDefault, tunnelSideSock, 0);
  CFRunLoopAddSource(CFRunLoopGetMain(), tunSocketSource, kCFRunLoopDefaultMode);
   
  CFRelease(tunSocketSource);

After I post the question, I had printed a lot of debug message in my code.
Then I found that CFRunLoop seems to which doesn't work correctly. It has to call the callback function when these events
Code Block
kCFSocketDataCallBack, kCFSocketReadCallBack, kCFSocketWriteCallBack
are getting hit.

But nothing happed.

My code run a main loop infinitely (But it is interruptable). Is it possible to block the CFRunLoop's main loop?


No problem. Where are you trying to access this run loop from? Is it inside your Network Extension? If so, a NetworkExtension does do not have guaranteed access to a run loop. See more on this from Quinns explanation here.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Thank you Matt.

But It's a little bit weird to me. Maybe I am misunderstanding.

If Network Extension run inside App Extension, It's working fine. The Run Loop is running normally.
I already confirmed it. Before I try to distribute the app with System Extension I was developing the app with Network Extension which is running in App Extension.

I assumed that the Run Loop is running asynchronously in Network Extension which inside App Extension. If Network Extension reside in System Extension, the Run Loop is running synchronously. Is that possible?
The routing table's behavior got weird after Network Extension runs in System Extension
 
 
Q