Btw, The issue can be stopped after restarting the bluetoothd since the XPC connection has been reset.

The crash is caused by an uncaught exception in com.apple.usernotifications.UNUserNotificationServiceConnection.call-out queue. It is essential to inspect your code to identify and address any instances where exceptions might be thrown.

Thanks @eskimo. "The SystemExtensions framework is intended to be used by a GUI app to load and unload system extensions in response to user actions. It’s not meant to be called from a command-line tool, regardless of how that command-line tool is packaged. Calling it from a command-line tool is unsupported and may yield weird results." Actually, the "activate/deactivate" commands are supported in my GUI app itself, it's not another command-line tool. What I did in the upgrade is to use install script to call my GUI app with parameter, may I know that is not supported?

Thanks @eskimo. "If the extension is already active, the request succeeds in short order, without significant delay or user interaction. If you request activation of a new version of an already-active extension, the system prompts the user to resolve the conflict before proceeding." I also did not see any system prompts for this case. In production: My main app can accept a external parameter to do activationRequestForExtension of the system extension. During the whole App upgrade, in the script of installer would call my main app with "activate" parameter to activate the new system extension. The log of sysextd and nesessionmanager said the system extension has been activated, but it was not running unless I restarted the nesessionmanager or reboot or deactivated it first and activated it again. default 23:36:25.429287+0800 sysextd setting +x bit on executable in bundle file:///Library/SystemExtensions/0B50E56F-108F-4F48-B9D0-26788FBB7817/com.android2test.sysext.systemextension/ default 23:36:25.429610+0800 sysextd notifying categories that extension com.android2test.sysext will start default 23:36:25.429687+0800 sysextd returning cdhash for local arch x86_64 of extension com.android2test.sysext default 23:36:25.430012+0800 sysextd starting extension com.android2test.sysext via owning category com.apple.system_extension.network_extension default 23:36:25.430049+0800 sysextd returning cdhash for local arch x86_64 of extension com.android2test.sysext default 23:36:25.431545+0800 sysextd extension point confirmed that extension com.android2test.sysext is runnable default 23:36:25.431587+0800 sysextd changing state of extension com.android2test.sysext to activated_enabled default 23:36:25.431615+0800 sysextd extension *** com.android2test.sysext (1.1.0/1.1.0) advancing state from activated_enabling to activated_enabled default 23:36:25.445482+0800 sysextd observer for 'com.android2test.app' reached success: activated_enabled Not every time, it's not 100% can be reproducible. Btw, if I set the system extension developer on(SIP disabled), then I can do multiple system extension activation with the same version. In this way, I can see the similar issue more easier.

Thanks @eskimo, I see the main issue(my extension can not be relaunched on both macOS 12 and 13 automatically after multiple activations without reboot, reboot can solve the issue as well) in both production(activate system extension again during the upgrade product) and developer machines(can reproduce by multiple activations). And both log of sysextd and nesessionmanager to show the system extension should be launched, but not. Btw, I noticed that if I activate system extension multiple times, the old ones would be terminated by sysextd, so looks no need to do deactivation before activate it?

I found below way to restart the nesessionmanager for question 2: sudo launchctl stop com.apple.nesessionmanager sudo launchctl start com.apple.nesessionmanager

Thanks, @meaton. What I am asking is for CFNetworkExecuteProxyAutoConfigurationURL(https://developer.apple.com/documentation/cfnetwork/1426392-cfnetworkexecuteproxyautoconfigu?language=objc), such as below code: CFRunLoopSourceRef source(CFNetworkExecuteProxyAutoConfigurationURL(pacURLRef, targetURLRef, callback, &context)); It would download a PAC and executes it. However, looks like it would not download PAC for each invocation. I think there might have a cache related to CFNetwork in its implementation. I want it to send a new request to download PAC for each invocation instead of using a cached one. So far I can reach that goal by implementing the download PAC part and feeding its content to CFNetworkExecuteProxyAutoConfigurationScript. Not sure if there is a simple way to disable the "cache" for API CFNetworkExecuteProxyAutoConfigurationURL or not.

Instead of traversing certs to check, I just found a API which can help to check customized cert installed and trusted: SecTrustCreateWithCertificates.

Thanks, @eskimo for your help. Yes, I still see the leaks caused by cycle reference? Report Version: 7 Analysis Tool:  /usr/bin/leaks Physical footprint:     2720K Physical footprint (peak): 2720K ---- leaks Report Version: 4.0 Process 43967: 787 nodes malloced for 85 KB Process 43967: 8 leaks for 800 total leaked bytes.   8 (800 bytes) ROOT CYCLE: <CFRunLoopSource 0x600003034000> [192]     7 (608 bytes) ROOT CYCLE: 0x600003c30000 [224]      CYCLE BACK TO <CFRunLoopSource 0x600003034000> [192]      5 (352 bytes) ROOT CYCLE: 0x600002b34a80 [128]        2 (80 bytes) ROOT CYCLE: 0x600000536c80 [32]         1 (48 bytes) ROOT CYCLE: <__NSMallocBlock__ 0x600000b34c00> [48] CFNetwork <unknown-symbol> 0x7ff819c39000 + 1605859 0x7ff819dc10e3        1 (32 bytes) ROOT CYCLE: 0x600000536ce0 [32]        1 (112 bytes) <NSURL 0x600002e344d0> [112] "http://example.com/"      1 (32 bytes) 0x600000538140 [32] Btw: If I use __bridge_transfer to replace the CFBridgingRelease in your code, then I get more leaks including the proxies. CFArrayRef proxiesArray = CFNetworkCopyProxiesForAutoConfigurationScript((__bridge CFStringRef)pac, (__bridge CFURLRef) url, &err); NSArray *proxies = (__bridge_transfer NSArray *)proxiesArray; more leaks: Report Version: 7 Analysis Tool:  /usr/bin/leaks Physical footprint:     2764K Physical footprint (peak): 2764K ---- leaks Report Version: 4.0 Process 44061: 786 nodes malloced for 85 KB Process 44061: 17 leaks for 1232 total leaked bytes.   17 (1.20K) ROOT CYCLE: <CFRunLoopSource 0x600003888000> [192]     16 (1.02K) ROOT CYCLE: 0x60000348c000 [224]      CYCLE BACK TO <CFRunLoopSource 0x600003888000> [192]      5 (352 bytes) ROOT CYCLE: 0x60000239c280 [128]        2 (80 bytes) ROOT CYCLE: 0x600000d9c220 [32]         1 (48 bytes) ROOT CYCLE: <__NSMallocBlock__ 0x60000039c000> [48] CFNetwork <unknown-symbol> 0x7ff819c39000 + 1605859 0x7ff819dc10e3        1 (32 bytes) ROOT CYCLE: 0x600000d9c280 [32]        1 (112 bytes) <NSURL 0x6000026980e0> [112] "http://example.com/"      9 (432 bytes) <CFArray 0x600001884200> [64] item count: 2        5 (240 bytes) <NSDictionary 0x6000018841c0> [64] item count: 3         1 (48 bytes) <CFString 0x600000384060> [48] length: 21 "kCFProxyPortNumberKey"         1 (48 bytes) <CFString 0x6000003840c0> [48] length: 16 "kCFProxyTypeHTTP"         1 (48 bytes) _list --> <CFString 0x600000384090> [48] length: 19 "kCFProxyHostNameKey"         1 (32 bytes) <CFString 0x600000d84060> [32] length: 11 "192.168.1.2"        3 (128 bytes) <NSDictionary 0x600000d84080> [32]         1 (48 bytes) _key --> <CFString 0x600000384030> [48] length: 15 "kCFProxyTypeKey"         1 (48 bytes) _obj --> <CFString 0x6000003840f0> [48] length: 16 "kCFProxyTypeNone"      1 (32 bytes) 0x600000d840a0 [32] What's the difference between __bridge_transfer and CFBridgingRelease? Both are used to transfer ownership of a Core Foundation object to an Objective-C object, right?

Thanks eskimo. Could we do: reduce the first "CONNECT" by adding a "Proxy-Authorization" header if we already got the credential? it's ok for the 2 "CONNECTs" for the 1st request, but how about the following requests? Do we have the same 2 "CONNECTs" for each request to the same proxy server? is there any way in "NSURLSession" to reuse the credential from 1st request for the following request? If we use CFNetwork, then we can use CFHTTPMessageApplyCredentialDictionary to set the credential which is from the 1st request for the following request to avoid 2 "CONNECTS".

For CFNetwork, we could set credentials by CFHTTPMessageApplyCredentialDictionary before the request if we got it already.

Thanks a lot, @eskimo. Please refer to FB9942875.

just found if any variable even primitive types captured by the block of setTerminationHandler, then there is a leak: void testLeaks() {    int abc = 123; NSTask* task = [[NSTask alloc] init]; [task setTerminationHandler:^(NSTask* inTask) { NSLog(@"setTerminationHandler %d %d", inTask.terminationStatus, abc);    }];    [task release]; } The output is the same: Process 10635: 593 nodes malloced for 59 KB Process 10635: 1 leak for 48 total leaked bytes. 1 (48 bytes) ROOT LEAK: <__NSMallocBlock__ 0x7f935ed09690> [48] testLeaks2 invocation function for block in testLeaks() 0x10a6a27e0

Hi @eskimo, I verified this issue can not be reproduced on 12.0.1. Looks Apple fixed it:)

I will do it, thanks a lot! @eskimo. Btw: I feel there is a security concern if a process is muted by path, let's say if I muted a trusted process by path, however, the process might be replaced by a totally different one with the same path, then the events of the "new" one also are muted, right?