Inexplicable difference in sandbox behavior targeting different versions of app

I'm debugging an odd situation wherein my app MarsEdit, which is sandboxed but has entitlements to send Apple events to BBEdit, is blocking from sending those events only with the newest version of BBEdit (13.5). For example, sending events to BBEdit 13.1

I have debugged the problem enough to learn that the failure occurs in the Apple events infrastructure, attempting to locate and open a port for the delivery of events. In the working case, I witness the following debug console message:

SUCCESSFULLY found another process by bundleID using the new appleeventsd server, "com.barebones.bbedit", port=( port:124783/0x1e76f rcv:0,send:2,d:0 limit:0)

Whereas in the broken case, I see:

debug 22:57:10.873976-0400 MarsEdit Couldn't find a process with pid 11325
SUCCESSFULLY found another process by bundleID using the new appleeventsd server, "com.barebones.bbedit", port=(null machport)

My best guess is that BBEdit 13.5 is being made "invisible" to MarsEdit based on a failure to apply the entitlement MarsEdit has to send Apple events to "com.barebones.bbedit", when targeting BBEdit 13.5.

I have confirmed that disabling the sandbox for my app causes the event delivery to succeed. I have also reproduced the error, which culminates in a -600 OSStatus error from AESend, in a trivial sandboxed test app which just tries to send an event to BBEdit.

Anybody have any ideas why BBEdit 13.5 might cause the system to exhibit this nuanced difference in behavior? It kind of smells like a "linked on or after" behavioral difference, which might make sense given the BBEdit 13.5 release seems to have been built on and linked with a newer SDK than previous releases.

Here is the complete source code for a sample app's App Delegate that exhibits the bug. Create a new, sandboxed Mac app with this as the app delegate source if you interested to reproduce the problem. Make sure to add a com.apple.security.temporary-exception.apple-events entitlement for com.barebones.bbedit.

Code Block
- (NSAppleEventDescriptor*) targetDescriptorAppWithIdentifier:(NSString*)bundleIdentifier
{
// Prefer a running instance
NSRunningApplication* runningApp = [NSRunningApplication runningApplicationsWithBundleIdentifier:bundleIdentifier].firstObject;
if (runningApp == nil) {
NSWorkspace *workspace = [NSWorkspace sharedWorkspace];
NSURL* targetURL = [workspace URLForApplicationWithBundleIdentifier:bundleIdentifier];
if (targetURL != nil) {
NSDictionary* launchConfig = @{};
NSError* launchError = nil;
runningApp = [workspace launchApplicationAtURL:targetURL options:0 configuration:launchConfig error:&launchError];
}
}
if (runningApp != nil) {
return [NSAppleEventDescriptor descriptorWithProcessIdentifier:runningApp.processIdentifier];
}
return nil;
}
- (NSAppleEventDescriptor*) appleEventForOpeningFileAtPath:(NSString*)filePath withApplicationBundleID:(NSString*)appBundleID displayingCustomPathString:(NSString*)customPathString
{
NSAppleEventDescriptor* targetDescriptor = [self targetDescriptorAppWithIdentifier:appBundleID];
NSAppleEventDescriptor* openFileEvent = [NSAppleEventDescriptor appleEventWithEventClass:kCoreEventClass eventID:kAEOpenDocuments targetDescriptor: targetDescriptor returnID: kAutoGenerateReturnID transactionID:kAnyTransactionID];
// Dont need actual file parameters to exhibit the connection
return openFileEvent;
}
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
AEDesc reply = {typeNull, NULL};
NSAppleEventDescriptor* openFileEvent = [self appleEventForOpeningFileAtPath:@"/tmp/hello" withApplicationBundleID:@"com.barebones.bbedit" displayingCustomPathString:@"HI"];
OSStatus status = AESend([openFileEvent aeDesc], &reply, kAENoReply, kAENormalPriority, kAEDefaultTimeout, NULL, NULL);
NSLog(@"Got AESend status %d", status);
}


If you launch BBEdit 13.5 so that it is running while the test app runs, you will see a -600 error from AESend, whereas if you launch BBEdit 13.1 so that it is running, you will see a noErr.
Yeah, that’s super weird. I can reproduce the problem myself (10.15.6) and in the system log I see these two entries:

Code Block
type: debug
time: 2020-10-18 20:45:33.913127 +0100
process: appleeventsd
subsystem: com.apple.appleevents
category: main
message: LOOKUP: pid=0 asn=0x-0 name=[ NULL ] bundleID="com.barebones.bbedit" signature=0x00000000 session=186aa
type: debug
time: 2020-10-18 20:45:33.913194 +0100
process: appleeventsd
subsystem: com.apple.appleevents
category: main
message: reply = {success=false, }


with nothing in between. It’s not at all clear why the lookup failed, especially as all the standard reasons for failure do log.

Regardless, of what else you do, I think this is bugworthy. Please post your bug number, just for the record.

Beyond that, my only suggestion is that you open a DTS tech support incident so I can spend more time researching a workaround.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"
Filed FB8811175, and you've got mail. :-)

Filed FB8811175

Thanks.

and you've got mail. :-)

So have you!

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"
For those following along at home, this error was caused by BBEdit 13.5 having a large Info.plist. This hits an arbitrary 50 KiB limit in the code signing subsystem which then triggers a problem in the Apple event subsystem. We’re using FB8811175 to track an OS-level fix for that but, for the moment, if you hit this problem then the workaround is to limit the size of your Info.plist.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"
Inexplicable difference in sandbox behavior targeting different versions of app
 
 
Q