Hi Matt, SampleGUI.app is not a container app for system extension. Here we bundled SampleSysExtHost.app (container app for extension) inside SampleGUI.app as below: ** /Applications/SampleGUI.app/Contents/Applications/SampleSysExtHost.app ** and container app structure is as below: SampleSysExtHost.app Contents/ MacOS/ SampleSysExtHost _CodeSignature/ embedded.provisionprofile Info.plist Library/ SystemExtensions/ my.sysext.bundleid.systemextension PkgInfo Resources/ Based on your input on bundling issue, I did an experiment where container app is not bundled in SampleGUI.app and distributed it along side SampleGUI.app that will be installed as /Applications/SampleSysExtHost.app. With this change, the issue is not resolved. Experiment details given below: Scenario: Installed version 4.3.2.10 as /Applications/SampleSysExtHost.app Upgraded to 4.3.2.11 . System extension upgrrade failed. Observations: On BIG SUR 11.6 and 12.4: sysexxtd log shows: default 12:11:04.335985+0530 sysextd /Applications/SampleSysExtHost.app/Contents/Library/SystemExtensions/my.sysext.bundleid.systemextension: package type not `DEXT` default 12:11:04.335985+0530 sysextd /Applications/SampleSysExtHost.app/Contents/Library/SystemExtensions/my.sysext.bundleid.systemextension: entitlement `com.apple.developer.endpoint-security.client` not present or not true default 12:11:04.336046+0530 sysextd activateDecision found existing entry of same version: state activated_enabled, ID D888D9B7-53B5-4728-9C16-E3294A73BA3F default 12:11:04.336103+0530 sysextd initial activation decision: requestAppReplaceAction() default 12:11:04.336127+0530 sysextd notifying client of activation conflict default 12:11:04.337141+0530 sysextd client approved continuing upgrade for my.sysext.bundleid default 12:11:04.337215+0530 sysextd attempting to realize properties with identifier my.sysext.bundleid default 12:11:04.337300+0530 sysextd UNIX error exception: 3 default 12:11:04.340020+0530 sysextd UNIX error exception: 3 default 12:11:04.344399+0530 sysextd sysextd/daemon_ipc_nsxpc.swift:16: Fatal error: unable to extract client info from connection Crashed at same place: Thread 1 Crashed:: Dispatch queue: sysextd.extension_manager 0  libswiftCore.dylib       0x00007fff2c899367 _assertionFailure(_:_:file:line:flags:) + 1767 1  sysextd             0x000000010aab074c 0x10aa7c000 + 214860 2  sysextd             0x000000010aab338d 0x10aa7c000 + 226189 3  sysextd             0x000000010aab0fbf 0x10aa7c000 + 217023 4  sysextd             0x000000010aab101f 0x10aa7c000 + 217119 5  com.apple.Foundation      0x00007fff213a0c86 __NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S1__ + 10 6  com.apple.Foundation      0x00007fff2134bb88 -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] + 2271 7  com.apple.Foundation      0x00007fff21303039 message_handler + 206 8  libxpc.dylib          0x00007fff20200c24 _xpc_connection_call_event_handler + 56 9  libxpc.dylib          0x00007fff201ffa9b _xpc_connection_mach_event + 938 10 libdispatch.dylib        0x00007fff203108a6 _dispatch_client_callout4 + 9 11 libdispatch.dylib        0x00007fff20327aa0 _dispatch_mach_msg_invoke + 444 12 libdispatch.dylib        0x00007fff20316493 _dispatch_lane_serial_drain + 263 13 libdispatch.dylib        0x00007fff203285e2 _dispatch_mach_invoke + 484 14 libdispatch.dylib        0x00007fff20316493 _dispatch_lane_serial_drain + 263 15 libdispatch.dylib        0x00007fff203170ad _dispatch_lane_invoke + 366 16 libdispatch.dylib        0x00007fff20320c0d _dispatch_workloop_worker_thread + 811 17 libsystem_pthread.dylib     0x00007fff204b745d _pthread_wqthread + 314 18 libsystem_pthread.dylib     0x00007fff204b642f start_wqthread + 15 Later, On big sur 11.6, I even observed that the new extension is activated with user approval, leading to 2 system extensions with same bundle ID in state activated enabled. The problems with this state are: We can not deactivate older version system extension even with system extension API request. The only way is to disable SIP and deactivate using systemextensionsctl reset. QUES: When SIP dependency will be removed to deactivate system extension using systemextensionsctl ? QUES: Are you planning to add ability to systemextensionsctl to deactivate system extension based on bundleId and version? It will be helpful to deactivate stale older version system extensions as explained in my scenario. So the issue still occurs even when container app distributed as separate app => this clarifies no issue with existing bundling..right? Now I am wondering what is causing this issue? Can any of the following cause this upgrade issue? Above log says the entitlement com.apple.developer.endpoint-security.client not present or not true for system extension. Can this cause issue while upgrading? 'systemextensionsctl list' command output shows the system extension version as 4.3.2.123/4.3.2.123. This versioning can cause issue?

There was a typo in above container app details. Please find the container app structure below: SampleSysExtHost.app Contents/ MacOS/ SampleSysExtHost _CodeSignature/ embedded.provisionprofile Info.plist Library/ SystemExtensions/ my.sysext.bundleid.systemextension PkgInfo Resources/

Thanks Matt for your point. If this is the case, I am wondering how the system allowed the system extension to be installed and loaded for the first time. What I mean is, why and how this restriction applies only at the time of upgrade? My container app looks like this: EncryptionProxyHost.app Contents/ MacOS/ SampleGUI _CodeSignature/ embedded.provisionprofile Info.plist Library/ SystemExtensions/ my.sysext.bundleid.systemextension PkgInfo Resources/ Right now, the above container app is bundled inside other app at /Applications/SampleGUI.app/Contents/Applications/, and system allowed installation and loading of system extension without reporting any issue about location, but failed at the time of upgrade. So if I move the container app to location /Applications/SampleGUI.app/Contents/Helpers/, will it be allowed for upgrade? Apart from this, does SystemExtensions framework any API to uninstall/deactivate system extension based on the combination of bundle ID and version?

Because of this upgrade issue, we are unable to uninstall the system extension. Deactivation request is failing. The only remaining way to uninstall the system extension is disabling SIP which should not be done in general. Can you please have a look at the above the signing related information and let us know what to do next.

Sharing the details again: **ContainerApp codesign output:** devs-iMac:Applications dev$ codesign -d --entitlements - SampleSysExtHost.app Executable=/Applications/SampleGUI.app/Contents/Applications/SampleSysExtHost.app/Contents/MacOS/SampleSysExtHost <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>com.apple.developer.networking.networkextension</key> <array> <string>app-proxy-provider-systemextension</string> </array> <key>com.apple.developer.system-extension.install</key> <true/> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> <key>com.apple.security.get-task-allow</key> <false/> </dict> </plist> **ContainerApp provisioning profile output:** devs-iMac:Contents dev$ security cms -D -i embedded.provisionprofile  <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>AppIDName</key> <string>[Profile AppID Name]</string> <key>ApplicationIdentifierPrefix</key> <array> <string>[TEAM_ID]</string> </array> <key>CreationDate</key> <date>[DATE]</date> <key>Platform</key> <array> <string>OSX</string> </array> <key>IsXcodeManaged</key> <false/> <key>DeveloperCertificates</key> <array> <data>[DATA]</data> </array> <key>Entitlements</key> <dict> <key>com.apple.developer.system-extension.install</key> <true/> <key>com.apple.developer.networking.networkextension</key> <array> <string>packet-tunnel-provider-systemextension</string> <string>app-proxy-provider-systemextension</string> <string>content-filter-provider-systemextension</string> <string>dns-proxy-systemextension</string> <string>dns-settings</string> </array> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>keychain-access-groups</key> <array> <string>[TEAM_ID].*</string> </array> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> </dict> <key>ExpirationDate</key> <date>[DATE</date> <key>Name</key> <string>[PROFILE NAME]</string> <key>ProvisionsAllDevices</key> <true/> <key>TeamIdentifier</key> <array> <string>[TEAM_ID]</string> </array> <key>TeamName</key> <string>[TEAM_NAME]</string> <key>TimeToLive</key> <integer>6570</integer> <key>UUID</key> <string>[UUID]</string> <key>Version</key> <integer>1</integer> </dict> </plist> **System extension codesign output:** devs-iMac:SystemExtensions dev$ codesign -d --entitlements - my.sysext.bundleid.systemextension <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>com.apple.developer.networking.networkextension</key> <array> <string>app-proxy-provider-systemextension</string> </array> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> <key>com.apple.security.application-groups</key> <array> <string>[TEAM_ID].com.test.appgroup</string> </array> <key>com.apple.security.get-task-allow</key> <false/> </dict> </plist> **System extension provisioning profile output:** devs-iMac:Contents dev$ security cms -D -i embedded.provisionprofile <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>AppIDName</key> <string>[NAME]</string> <key>ApplicationIdentifierPrefix</key> <array> <string>[TEAM_ID]</string> </array> <key>CreationDate</key> <date>[DATE]</date> <key>Platform</key> <array> <string>OSX</string> </array> <key>IsXcodeManaged</key> <false/> <key>DeveloperCertificates</key> <array> <data>[DATA]</data> </array> <key>Entitlements</key> <dict> <key>com.apple.developer.networking.networkextension</key> <array> <string>packet-tunnel-provider-systemextension</string> <string>app-proxy-provider-systemextension</string> <string>content-filter-provider-systemextension</string> <string>dns-proxy-systemextension</string> <string>dns-settings</string> </array> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>keychain-access-groups</key> <array> <string>[TEAM_ID].*</string> </array> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> </dict> <key>ExpirationDate</key> <date>[DATE]</date> <key>Name</key> <string>[PROFILE_NAME]</string> <key>ProvisionsAllDevices</key> <true/> <key>TeamIdentifier</key> <array> <string>[TEAM_ID]</string> </array> <key>TeamName</key> <string>[TEAM_NAME]</string> <key>TimeToLive</key> <integer>6570</integer> <key>UUID</key> <string>[UUID]</string> <key>Version</key> <integer>1</integer> </dict> </plist>

Thanks Matt for the reply. If the code signing is problem, how the installation has been successful? However, for the analysis, I am sharing further information below: System extension installation location: /Applications/SampleGUI.app/Contents/Applications/SampleSysExtHost.app/Contents/Library/SystemExtensions/my.sysext.bundleid.systemextension SampleSysExtHost.app ===> system extension container app my.sysext.bundleid.systemextension ===> network system extension. Please find the container app and system extension code signing and entitlements details in attached file: CodeSigning Details

Hi Eskimo, Thanks for your reply. We still observe the issue even after restart. Sharing some more information here: The current output of 'systemextensionsctl list' looks like this (same as before update too) admin@admins-MacBook-Pro-2 ~ % systemextensionsctl list                                            1 extension(s) --- com.apple.system_extension.network_extension enabled active teamID bundleID  (version) name [state] '* *   ABC my.sysext.bundleid (4.3.2.123/4.3.2.123) MySystemExtension [activated enabled] Suppose we are trying to update to 4.3.2.567 and it is failing.

Hi,  I am observing the similar crash in sysextd while updating a network system extension on macOS12.4. The update failed, I still see old system extension. Please find the crash report below: Translated Report (Full Report Below) Process: sysextd [1214] Path: /System/Library/Frameworks/SystemExtensions.framework/Versions/A/Helpers/sysextd Identifier: sysextd Version: ??? Code Type: X86-64 (Native) Parent Process: launchd [1] User ID: 0 Date/Time: 2022-06-21 18:22:30.1240 +0530 OS Version: macOS 12.4 (21F79) Report Version: 12 Bridge OS Version: 6.5 (19P5071) Anonymous UUID: 492561C8-C69A-A8BF-3340-633DA3ABC374 Time Awake Since Boot: 1600 seconds System Integrity Protection: enabled Crashed Thread: 1 Dispatch queue: sysextd.extension_manager Exception Type: EXC_BAD_INSTRUCTION (SIGILL) Exception Codes: 0x0000000000000001, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY Termination Reason: Namespace SIGNAL, Code 4 Illegal instruction: 4 Terminating Process: exc handler [1214] Thread 0: 0 libsystem_kernel.dylib 0x7ff81cd1e1aa __semwait_signal_nocancel + 10 1 libsystem_c.dylib 0x7ff81cc4db18 nanosleep$NOCANCEL + 185 2 libsystem_c.dylib 0x7ff81cc4da25 sleep$NOCANCEL + 41 3 libdispatch.dylib 0x7ff81cbaf75f _dispatch_queue_cleanup2 + 158 4 libsystem_pthread.dylib 0x7ff81cd53567 _pthread_tsd_cleanup + 487 5 libsystem_pthread.dylib 0x7ff81cd55b89 _pthread_exit + 70 6 libsystem_pthread.dylib 0x7ff81cd5333d pthread_exit + 42 7 libdispatch.dylib 0x7ff81cbaafb9 dispatch_main + 96 8 sysextd 0x10d40a2a1 0x10d401000 + 37537 9 sysextd 0x10d4098f9 0x10d401000 + 35065 10 dyld 0x1137ef51e start + 462 Thread 1 Crashed:: Dispatch queue: sysextd.extension_manager 0 libswiftCore.dylib 0x7ff82a22e245 assertionFailure(:_:file:line:flags:) + 421 1 sysextd 0x10d436803 0x10d401000 + 219139 2 sysextd 0x10d43a407 0x10d401000 + 234503 3 sysextd 0x10d437308 0x10d401000 + 221960 4 sysextd 0x10d43736f 0x10d401000 + 222063 5 Foundation 0x7ff81dcde088 NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S1 + 10 6 Foundation 0x7ff81dc88738 -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] + 2347 7 Foundation 0x7ff81dc3f31e message_handler + 206 8 libxpc.dylib 0x7ff81ca94b70 _xpc_connection_call_event_handler + 56 9 libxpc.dylib 0x7ff81ca93956 _xpc_connection_mach_event + 1413 10 libdispatch.dylib 0x7ff81cb9e3b1 _dispatch_client_callout4 + 9 11 libdispatch.dylib 0x7ff81cbb7041 _dispatch_mach_msg_invoke + 445 12 libdispatch.dylib 0x7ff81cba41cd _dispatch_lane_serial_drain + 342 13 libdispatch.dylib 0x7ff81cbb7b77 _dispatch_mach_invoke + 484 14 libdispatch.dylib 0x7ff81cba41cd _dispatch_lane_serial_drain + 342 15 libdispatch.dylib 0x7ff81cba4dfd _dispatch_lane_invoke + 366 16 libdispatch.dylib 0x7ff81cbaeeee _dispatch_workloop_worker_thread + 753 17 libsystem_pthread.dylib 0x7ff81cd51fd0 _pthread_wqthread + 326 18 libsystem_pthread.dylib 0x7ff81cd50f57 start_wqthread + 15 Thread 2: 0 libsystem_kernel.dylib 0x7ff81cd1e1d2 __sigsuspend_nocancel + 10 1 libdispatch.dylib 0x7ff81cbaf81b _dispatch_sigsuspend + 36 2 libdispatch.dylib 0x7ff81cbaf7f7 _dispatch_sig_thread + 49 Thread 3: 0 libsystem_pthread.dylib 0x7ff81cd50f48 start_wqthread + 0 Thread 4: 0 libsystem_pthread.dylib 0x7ff81cd50f48 start_wqthread + 0 Thanks, Ravi

Thanks Matt for your immediate response. We use NEAppProxyProvider as we still use Xcode11.x. Per your suggestion, we will try using NETransparentProxyProvider and check the behaviour.

Opened feedback request, ID is FB8935382

Thanks Matt for the reply! When we add/update IMAP account settings and save, then the flow is going through accountsd (either 993 or 143) which can not be captured by transparent proxy. Once the mail account is configured and saved (either it is 143 or 993), if we send a mail from Mail.app, then Mail app is making connection to Mail Server and that is being captured. So in brief, Mail.app makes connection (any SMTP or IMAP ports) => proxy catches that flow. Steps to reproduce the specific scenario are: #1: Add an IMAP mail account on IMAP SSL port 993 and SMTP non-SSL port 25 and save it. #2: Install transparent proxy and it is running with the configured network rules to catch any SMTP and IMAP traffic. #3. Send mail from Mail.app and it gets captured. #4: While transparent proxy is up and running, open IMAP mail account settings and edit settings of IMAP from SSL to nonSSL and port from 993 to 143 and click Save. When Save is clicked, the connection which goes out is from accountsd and not from Mail.app. This is not being captured. My network rules are as below: (NENetworkRule *)createNeRuleFor:(NSString *)port { [[NENetworkRule alloc] initWithRemoteNetwork:[NWHostEndpoint endpointWithHostname:@"0.0.0.0" port:port]                                            remotePrefix:0                                            localNetwork:nil                                            localPrefix:0                                            protocol:NENetworkRuleProtocolTCP                                            direction:NETrafficDirectionOutbound]; } settings.includedNetworkRules = @[         [self createNeRuleFor:@"25"],         [self createNeRuleFor:@"587"],         [self createNeRuleFor:@"110"],         [self createNeRuleFor:@"995"],         [self createNeRuleFor:@"143"],         [self createNeRuleFor:@"993"],     ]; Query#1: Is there any restriction on the ability of network transparent proxies on catching network traffic from some system processes such as accountsd process?

I have couple of queries: Is there any way we can disable or grey out the connect/disconnect option for transparent proxy provider in network preferences? The transparent proxy provider is shown as VPN application in network preferences. Will this be impacted when other VPNs are installed and running on the system?

Matt, You mentioned that "The container app and Network System Extension for your NETransparentProxyProvider will also need a Sandbox". But in my sample, I have got a network system extension inside an app without sandbox. It is working as expected. One point to note here - disabled SIP on my system. Does disabling SIP lift the sandbox requirement of app and network system extension?

Thanks Matt for your reply. I got the looping issue resolved. I have a generic question on macOS network extension - transparent proxy provider. Is it a must to implement transparent proxy provider as a system extension inside an app? With this approach, we are observing 2 popups , one popup for system extension, one popup while adding VPN configuration (for monitoring network). The other issue is user can see the transparent proxy in the Network Preferences, which is weird. This is something different when compared to NKE behavior. We are looking for ways to avoid these 3 things. Can we avoid above popups and proxy inside network preferences? Before trying anything, I would like a guidance whether trying that approach worth or not. We are thinking to try implementing transparent proxy provider inside a launchdaemon? Can we implement transparent proxy provider inside a launchdaemon? or if system extension is the only option, then how can. we avoid above 2 popups and proxy visible in network preference?

I am also facing the same issue of getting into loop. I am not clear about the method of using audit token from flow's metadata. If you have a working example, can you please share. Is there any simple way of avoiding the capture of tcp flow coming from specific process(in our case it is launch agent).