NEPacketTunnelProvider consistently crashes on a few devices

We're running into this issue with two apps (one built using Xcode 11.0 in case it matters) that both have a packet tunnel provider extension. The packet tunnel provider extension will consistently crash only on a few devices before the NEPacketTunnelProvider class has a chance to be instantiated.


Console logs of the crash on an iOS 12.1 device:

debug 09:47:42.976623 +0530 NFNetworkExtension Found LSExtensionPoint <private> for identifier <private>
debug 09:47:42.979965 +0530 NFNetworkExtension -[PKPlugInCore setDictionaries:] <private>
debug 09:47:42.980335 +0530 NFNetworkExtension -[PKPlugInCore setupWithName:url:bundleInfo:info:uuid:] <private>
debug 09:47:42.982089 +0530 NFNetworkExtension Found LSExtensionPoint <private> for identifier <private>
debug 09:47:42.982419 +0530 NFNetworkExtension -[PKPlugInCore setDictionaries:] <private>
debug 09:47:42.982747 +0530 NFNetworkExtension -[PKPlugInCore resolveSDK] <private> merged with SDK
debug 09:47:42.983177 +0530 NFNetworkExtension -[PKPlugInCore infoKey:] CFBundleShortVersionString => <private>
debug 09:47:43.044194 +0530 NFNetworkExtension CFPrefsManagedSource<0x10230d490> (Domain: com.symantec.nfchild.dev.NFNetExtension, User: kCFPreferencesCurrentUser, ByHost: Yes, Container: (null), Contents Need Refresh: No) loaded: an empty base plist and no additional changes from the base plist
debug 09:47:43.044322 +0530 NFNetworkExtension CFPrefsManagedSource<0x10230d660> (Domain: com.symantec.nfchild.dev.NFNetExtension, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null), Contents Need Refresh: No) loaded: an empty base plist and no additional changes from the base plist
debug 09:47:43.044505 +0530 NFNetworkExtension CFPrefsPlistSource<0x10230db80> (Domain: com.symantec.nfchild.dev.NFNetExtension, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No) loaded: an empty base plist and no additional changes from the base plist
debug 09:47:43.044689 +0530 NFNetworkExtension CFPrefsManagedSource<0x10230df20> (Domain: kCFPreferencesAnyApplication, User: kCFPreferencesCurrentUser, ByHost: Yes, Container: (null), Contents Need Refresh: No) loaded: a new base plist and no additional changes from the base plist
debug 09:47:43.044861 +0530 NFNetworkExtension CFPrefsPlistSource<0x10230e020> (Domain: kCFPreferencesAnyApplication, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No) loaded: a new base plist and no additional changes from the base plist
debug 09:47:43.045084 +0530 NFNetworkExtension CFPrefsPlistSource<0x10230e0f0> (Domain: com.symantec.nfchild.dev.NFNetExtension, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null), Contents Need Refresh: No) loaded: an empty base plist and no additional changes from the base plist
debug 09:47:43.045727 +0530 NFNetworkExtension CFPrefsPlistSource<0x10230e1f0> (Domain: kCFPreferencesAnyApplication, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null), Contents Need Refresh: No) loaded: an empty base plist and no additional changes from the base plist
debug 09:47:43.045936 +0530 NFNetworkExtension looked up value <private> for key AppleLanguages in CFPrefsPlistSource<0x10230e020> (Domain: kCFPreferencesAnyApplication, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No) via CFPrefsSearchListSource<0x10230d230> (Domain: com.symantec.nfchild.dev.NFNetExtension, Container: (null))
debug 09:47:43.046063 +0530 NFNetworkExtension setting new value <private> for key <private> in CFPrefsSource<0x10230dfe0> (Domain: Volatile, User: , ByHost: No, Container: , Contents Need Refresh: No)
debug 09:47:43.046343 +0530 NFNetworkExtension setting new value <private> for key <private> in CFPrefsSource<0x10230dfe0> (Domain: Volatile, User: , ByHost: No, Container: , Contents Need Refresh: No)
debug 09:47:43.127001 +0530 NFNetworkExtension setting new value <private> for key <private> in CFPrefsSource<0x10230dfe0> (Domain: Volatile, User: , ByHost: No, Container: , Contents Need Refresh: No)
default 09:47:43.127668 +0530 NFNetworkExtension *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSDictionaryM setObject:forKey:]: object cannot be nil (key: E9220768-902D-467B-BC97-AC1912A14ACE)'
*** First throw call stack:
(0x1c88f9ea4 0x1c7ac9a50 0x1c8871384 0x1c87e4920 0x1c933fb5c 0x1c933f1e4 0x1c83316c8 0x1c8332484 0x1c82d9bd0 0x1c82da718 0x1c82e2eb8 0x1c85150dc 0x1c8517cec)


Stack trace:

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Triggered by Thread:  3


Last Exception Backtrace:
0   CoreFoundation                0x182f86d8c __exceptionPreprocess + 228 (NSException.m:166)
1   libobjc.A.dylib                0x1821405ec objc_exception_throw + 56 (objc-exception.mm:521)
2   CoreFoundation                0x182f1f750 _CFThrowFormattedException + 112 (CFObject.m:1946)
3   CoreFoundation                0x182e54c6c -[__NSDictionaryM setObject:forKey:] + 948 (NSDictionaryM.m:176)
4   Foundation                    0x1839664f8 -[_NSExtensionContextVendor _setPrincipalObject:forUUID:] + 100 (NSExtensionContext.m:362)
5   Foundation                    0x183965b84 __105-[_NSExtensionContextVendor _beginRequestWithExtensionItems:listenerEndpoint:withContextUUID... + 688 (NSExtensionContext.m:264)
6   libdispatch.dylib              0x182878aa0 _dispatch_call_block_and_release + 24 (init.c:994)
7   libdispatch.dylib              0x182878a60 _dispatch_client_callout + 16 (object.m:507)
8   libdispatch.dylib              0x1828829b4 _dispatch_queue_serial_drain$VARIANT$mp + 608 (inline_internal.h:2500)
9   libdispatch.dylib              0x1828832fc _dispatch_queue_invoke$VARIANT$mp + 336 (queue.c:5290)
10  libdispatch.dylib              0x182883cc8 _dispatch_root_queue_drain_deferred_wlh$VARIANT$mp + 340 (queue.c:5908)
11  libdispatch.dylib              0x18288c098 _dispatch_workloop_worker_thread$VARIANT$mp + 668 (source.c:2529)
12  libsystem_pthread.dylib        0x182****70 _pthread_wqthread + 860 (pthread.c:2218)
13  libsystem_pthread.dylib        0x182babb08 start_wqthread + 4


Thread 0 name:
Thread 0:
0   libsystem_kernel.dylib        0x00000001829ebde8 mach_msg_trap + 8
1   libsystem_kernel.dylib        0x00000001829ebc60 mach_msg + 72 (mach_msg.c:103)
2   CoreFoundation                0x0000000182f2ee40 __CFRunLoopServiceMachPort + 196 (CFRunLoop.c:2613)
3   CoreFoundation                0x0000000182f2c908 __CFRunLoopRun + 1568 (CFRunLoop.c:2969)
4   CoreFoundation                0x0000000182e4cda8 CFRunLoopRunSpecific + 552 (CFRunLoop.c:3245)
5   Foundation                    0x00000001838c1674 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 304 (NSRunLoop.m:367)
6   Foundation                    0x00000001839361a8 -[NSRunLoop(NSRunLoop) run] + 88 (NSRunLoop.m:389)
7   libxpc.dylib                  0x0000000182bf5a6c _xpc_objc_main + 516 (main.m:167)
8   libxpc.dylib                  0x0000000182bf7b40 xpc_main + 180 (init.c:1476)
9   Foundation                    0x0000000183af1fd4 -[NSXPCListener resume] + 180 (NSXPCListener.m:253)
10  PlugInKit                      0x0000000188602ec4 -[PKService run] + 712 (PKService.m:105)
11  PlugInKit                      0x0000000188602ab4 +[PKService main] + 56 (PKService.m:39)
12  PlugInKit                      0x0000000188602ee8 +[PKService _defaultRun:arguments:] + 20 (PKService.m:116)
13  Foundation                    0x0000000183ad8a98 NSExtensionMain + 64 (NSExtensionMain.m:23)
14  libdyld.dylib                  0x00000001828ddfc0 start + 4


Thread 1:
0   libsystem_kernel.dylib        0x0000000182a0dd78 __workq_kernreturn + 8
1   libsystem_pthread.dylib        0x0000000182babeb4 _pthread_wqthread + 928 (pthread.c:0)
2   libsystem_pthread.dylib        0x0000000182babb08 start_wqthread + 4


Thread 2:
0   libsystem_pthread.dylib        0x0000000182babb04 start_wqthread + 0 (pthread.c:2458)


Thread 3 name:
Thread 3 Crashed:
0   libsystem_kernel.dylib        0x0000000182a0d2e0 __pthread_kill + 8
1   libsystem_pthread.dylib        0x0000000182bae288 pthread_kill$VARIANT$mp + 376 (pthread.c:1484)
2   libsystem_c.dylib              0x000000018297bd0c abort + 140 (abort.c:91)
3   libc++abi.dylib                0x00000001821172c8 abort_message + 132 (abort_message.cpp:75)
4   libc++abi.dylib                0x0000000182117470 default_terminate_handler() + 304 (cxa_default_handlers.cpp:68)
5   libobjc.A.dylib                0x00000001821408d4 _objc_terminate() + 124 (objc-exception.mm:657)
6   libc++abi.dylib                0x000000018213137c std::__terminate(void (*)()) + 16 (cxa_handlers.cpp:66)
7   libc++abi.dylib                0x0000000182130ccc __cxa_throw + 132 (cxa_exception.cpp:130)
8   libobjc.A.dylib                0x0000000182140720 objc_exception_throw + 364 (objc-exception.mm:547)
9   CoreFoundation                0x0000000182f1f750 _CFThrowFormattedException + 112 (CFObject.m:1946)
10  CoreFoundation                0x0000000182e54c6c -[__NSDictionaryM setObject:forKey:] + 948 (NSDictionaryM.m:176)
11  Foundation                    0x00000001839664f8 -[_NSExtensionContextVendor _setPrincipalObject:forUUID:] + 100 (NSExtensionContext.m:362)
12  Foundation                    0x0000000183965b84 __105-[_NSExtensionContextVendor _beginRequestWithExtensionItems:listenerEndpoint:withContextUUID... + 688 (NSExtensionContext.m:264)
13  libdispatch.dylib              0x0000000182878aa0 _dispatch_call_block_and_release + 24 (init.c:994)
14  libdispatch.dylib              0x0000000182878a60 _dispatch_client_callout + 16 (object.m:507)
15  libdispatch.dylib              0x00000001828829b4 _dispatch_queue_serial_drain$VARIANT$mp + 608 (inline_internal.h:2500)
16  libdispatch.dylib              0x00000001828832fc _dispatch_queue_invoke$VARIANT$mp + 336 (queue.c:5290)
17  libdispatch.dylib              0x0000000182883cc8 _dispatch_root_queue_drain_deferred_wlh$VARIANT$mp + 340 (queue.c:5908)
18  libdispatch.dylib              0x000000018288c098 _dispatch_workloop_worker_thread$VARIANT$mp + 668 (source.c:2529)
19  libsystem_pthread.dylib        0x0000000182****70 _pthread_wqthread + 860 (pthread.c:2218)
20  libsystem_pthread.dylib        0x0000000182babb08 start_wqthread + 4


Entitlements from the PacketTunnelExtension binary using `codesign -d --entilements -- ...` show:

<key>com.apple.developer.networking.networkextension</key>
<array>
<string>packet-tunnel-provider</string>
</array>


Info.plist

<key>NSExtensionPointIdentifier</key>
<string>com.apple.networkextension.packet-tunnel</string>
<key>NSExtensionPrincipalClass</key>
<string>$(PRODUCT_MODULE_NAME).PacketTunnelProvider</string>


Any ideas on what's happening here?

NEPacketTunnelProvider consistently crashes on a few devices
 
 
Q