Just got a crash log from our QA testing on Monterey. The basic crash info is here, but I don't know what the termination reason means or how to look it up. Exception Type: EXC_CRASH (SIGKILL) Exception Codes: 0x0000000000000000, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY Termination Reason: Namespace ENDPOINTSECURITY, Code 2 The backtrace showed the crashing thread in a _pthread_cond_wait waiting for a signal that there is data to process. (This is for notify events, we queue them up for processing). We do retain the message, so I'm wondering if the queue is backing up (shouldn't be since the pthread_cond_wait only happens on an empty queue) and not releasing the messages quickly enough. But I don't know what Code 2 means to see if this idea is valid. Can someone point me to where I can look that up? Thanks

Hi all (and hopefully Quinn) I found this WWDC session today (https://developer.apple.com/videos/play/wwdc2021/10130/) dealing with MDM updates for Monterey. The part that caught my eye was the new RemovableSystemExtensions feature. From the video: In macOS Monterey, there is a new feature called RemovableSystemExtension. This will allow an app to deactivate its own system extension, for example, when the app uninstalls itself. With this feature, there will be no admin password required to remove the system extension. This may be useful in deployments where the Mac has no admin user. My question is, do I (we) have to do anything special in our system extension to take advantage of this or is it a free pass from MDM? I already do a call to deactivationRequestForExtension when we uninstall. And that works fine on Big Sur, but doesn't any more on Monterey. That's what led me down the path that found this session and other links. I know I'm not the only one suffering through the uninstall aspect of system extensions and really hope that this is the holy grail. Thanks

We have some code that was built with the 10.10 SDKs (don't ask, it just is). When it runs on Monterey and gets the version number, it gets 10.16.0, just like Big Sur. I was expecting 10.17.0 for a new version. Will Monterey get the minor version number bumped to not look like Big Sur? Thanks

Quinn, you've often suggested that to validate the other side of an XPC connection, we should use the audit token. But that's not available from the XPC object, whereas the PID is. So everyone uses the PID. While looking for something completely unrelated, I found this in the SecCode.h file OSStatus SecCodeCreateWithXPCMessage(xpc_object_t message, SecCSFlags flags, SecCodeRef * __nonnull CF_RETURNS_RETAINED target); Would this be the preferred way to do this now? At least from 11.0 and up. Like I said, I was looking for something completely unrelated and found this and don't have the cycles right now to try it. But it looks promising from the description and I wanted to check in with you about it in case you can say yes or no before I get a chance to test it. Thanks

When testing our Endpoint Security product on Big Sur beta 8 or 9, we found that if SIP is enabled our EndpointSecurity system extension hangs and is then killed by the kernel when another process runs "vmmap -w <PID of extension>". This happens when either ES_EVENT_TYPE_AUTH_OPEN or ES_EVENT_TYPE_AUTH_MMAP are submitted as events to subscribe to. No other events that we use cause this issue. If SIP is disabled, this does not occur. To make sure that it wasn't our code not returning quick enough, I tested with just AUTH_OPEN and have the client callback just return "es_respond_flags_result(client, msg, 0xffffffff, true);". I also disable pretty much every thing else that our extension does, like create an XPC listener for messages from our service, no other notify or auth subscriptions, just AUTH_OPEN and returning a flags result. This feels very much like my other ticket, FB7526331. We were given a work-around of subscribing to the AUTH_GET_TASK event and deny it if we were the target. I hope that there is something like that for this issue. Sorry about the weird formatting, not doing code messes up all the underscores. Quinn, before you ask, this is FB8783607

When an Endpoint Security client gets an auth message, one of the fields in that message is the deadline, as described in the ESMessage.h file as so:* @field deadline The Mach time before which an auth event must be responded to. * If a client fails to respond to auth events prior to the `deadline`, the client will be killed.What happens if the client gets the message and either the machine is put to sleep or the VM is suspended until past the deadline time and the client can't repsond? Will the kernel still kill the client? Should it be smarter and understand sleeping / suspension?I ask because I've seen many deaths upon waking my VM and seeing this dreaded message in the log."2020-03-10 10:50:08.284731-0700 0x430d Error 0x0 0 0 kernel: (EndpointSecurity) Client did not respond in appropriate amount of time (client pid: 773)"

Hi,I have the NSEndpointSecurityEarlyBoot flag set in plist for my ES extension. I see an entry in /Library/SystemExtensions/EndpointSecurity/.early_boot.plist that has a hash that corresponds to my extension in the /Library/SystemExtensions/db.plist file. According to the man page entry quoted below NSEndpointSecurityEarlyBoot Type: Boolean If set to TRUE, the ES subsystem will hold up all third party executions (anything that is not a platform binary) until all early boot ES extensions make their first subscription.my extension should load right after the Apple system extension, right? What I'm seeing is that it loads early, but other binaries are loaded before it, e.g., my launchd service code. The PID for the service code is 100 and the PID for the extension is 269 for the most recent test.That doesn't feel right to me based on the information above. My question is, 1) Is there something else I need to do to load early or, 2) am I intrepreting this wrong, or 3) is this a bug I should report?Thanks

Hi all (especially Quinn if you watching),Using everyone's favorite Endpoint Security demo code to narrow down a problem I've been seeing in my code, https://gist.github.com/Omar-Ikram/8e6721d8e83a3da69b31d4c2612a68ba (thank you, Omar), I've been able to reproduce a kill of the ES process.If I subscribe to either the ES_EVENT_TYPE_AUTH_OPEN or ES_EVENT_TYPE_AUTH_MMAP event, when I run a leaks command on the process, it hangs, then eventually dies with a Killed: 9 message. Those are the only 2 events I've found that do this, though my search hasn't been exhaustive. I am also guessing that there are other commands besides leaks that will do this, but I can reproduce this 100% of the time with leaks.I've tried using the async dispatch to make sure I'm not totally blocking the kernel, I've tried shortcutting the code to always just return ES_AUTH_RESULT_ALLOW to es_respond_auth_result without doing anything else, I've even considered consulting a witch doctor, but thought I'd ask here first. 🙂Is this a known issue? I hope it's not by design, this would give malware a way to kill an EndpointSecurity process so it could try to gain a foothold before the EndpointSecurity process could restart.If this is a problem, I'll write up a RADAR ticket and post the number here. If not, what is the suggested work-around?Thanks!Mike