Error when loading (some) Audio Units on macOS 12

The “developper can not be verified” message suggests you’re hitting a Gatekeeper block rather than something related to the hardened runtime.

Are you able to reproduce this yourself? Or are you only seeing it on this specific customer machine?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks for you reply. I tested one of these plugins on an intel iMac running macOS 10.15 without problem. Today, I also tested the same plugin on an intel iMac running macOS 13.2.1 and it opened normally. So, no, I wasn't able to reproduce the bug. But it could be due to the fact that the systems I used was different (unfortunately I don't have a machine running macOS 12 here). For Catalina, the result is probably related to the fact the system is less secure than Monterey, but for Ventura it's unclear if it's related to the plugin, the hardware or the system.

I also forgot to say that, during this beta testing session, we tried to disable Gate Keeper on the beta tester machine using the 'sudo spctl --master-disable' command, and that was successful. This seems to confirm your assumption. Knowing that, do you think there is a possible workaround using some other entitlements ?

dp

How does this failure manifest at the API level? When you trigger this error, does an API fail? If so, which one?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

The warning appears when trying to create an instance of any audio unit provided by some specific manufacturers and to open it in its editor (both things being merged as a single operation). The message body was followed by another sentence :

macOS cannot verify that this app does not contain any malicious software

and

<AppName> created this file today at 15:32

The file in question was a file called NSCreatedObjectFileImageFromMemory-p47UEwps stored in the temporary directory (-[NSURL temporaryDirectory]) of the host application.

The panel buttons proposed either to trash the file or to cancel. Whatever the button clicked, a second dialog appears with an exclamation mark in a red icon and the Fatal Error 100001 message, and the plugin never opened (but the host application didn't crash).

Here is a part of the crash report (if you want, I can send the full report to your email address):

Date/Time:             2023-08-24 15:04:45.7209 +0200
OS Version:            macOS 12.5.1 (21G83)
Report Version:        12
Bridge OS Version:     6.6 (19P6066)
Anonymous UUID:        0495D4E2-3107-8006-D560-7301560EB3CE

Sleep/Wake UUID:       2CFC336A-EB4A-4493-BB08-8CCCBA20E12F

Time Awake Since Boot: 73000 seconds
Time Since Wake:       596 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGBUS)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x0000000137c7cc34
Exception Codes:       0x0000000000000002, 0x0000000137c7cc34
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace SIGNAL, Code 10 Bus error: 10
Terminating Process:   exc handler [41378]

VM Region Info: 0x137c7cc34 is in 0x137bb1000-0x137cc9000;  bytes after start: 834612  bytes before end: 312267
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      mapped file                 135c56000-137773000    [ 27.1M] r--/rwx SM=COW  ...t_id=d8860195
      GAP OF 0x43e000 BYTES
--->  __TEXT                      137bb1000-137cc9000    [ 1120K] r-x/rwx SM=COW  ...le SSL 4000 G
      __DATA                      137cc9000-137cf5000    [  176K] rw-/rwx SM=COW  ...le SSL 4000 G

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread

0   bx_console SSL 4000 G         	       0x137bdf476 0x137bb1000 + 189558
1   bx_console SSL 4000 G         	       0x137c7e739 AUV2AdapterEntry + 439641
2   dyld                          	       0x1148ca334 dyld4::RuntimeState::notifyObjCInit(dyld4::Loader const*) + 170
3   dyld                          	       0x1148cff03 dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 167
4   dyld                          	       0x1148cffb2 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 108
5   dyld                          	       0x1148dee00 dyld4::APIs::dlopen_from(char const*, int, void*) + 592
6   CoreFoundation                	    0x7ff815cd6994 _CFBundleDlfcnLoadBundle + 149
7   CoreFoundation                	    0x7ff815d5094f _CFBundleLoadExecutableAndReturnError + 484
8   AudioToolboxCore              	    0x7ff81712efa5 APComponent_FromBundle_Loadable::ResolveFactoryFunction() + 67
9   AudioToolboxCore              	    0x7ff81712ded2 APComponent::newInstance(unsigned int, bool, void (OpaqueAudioComponentInstance*, int) block_pointer) + 126
10  AudioToolboxCore              	    0x7ff817223c90 instantiate(OpaqueAudioComponent*, unsigned int, bool, void (OpaqueAudioComponentInstance*, int) block_pointer) + 311
11  AudioToolboxCore              	    0x7ff81722402c __AudioComponentInstanceNew_block_invoke + 88
12  AudioToolboxCore              	    0x7ff8170b5479 Synchronously + 87
13  AudioToolboxCore              	    0x7ff817223ee2 AudioComponentInstanceNew + 183
14  <AppName>                  	       0x10f9274e3 mx::EditableDevice::instanciate(AudioComponentDescription*, unsigned short, unsigned char) + 77
15  <AppName>                  	       0x10f9273cf mx::EditableDevice::EditableDevice(mx::MixerBase*, AudioComponentDescription*, unsigned short, bool) + 101
16  <AppName>                  	       0x10f9087ab mx::Insert::Insert(mx::MixerBase*, AudioComponentDescription*, unsigned char, unsigned char, unsigned char) + 41
17  <AppName>                  	       0x10f90e957 mx::Track::insertEffectAtIndex(unsigned long long, unsigned char, unsigned char, unsigned char, unsigned char) + 559

Well, that’s weird. With regards this:

0   bx_console SSL 4000 G … 0x137bb1000 + 189558
1   bx_console SSL 4000 G … AUV2AdapterEntry + 439641
2   dyld                  … dyld4::RuntimeState::notifyObjCInit(dyld4::Loader const*) + 170

is bx_console SSL 4000 G the name of the plug-in? Or something from your app?

if you want, I can send the full report to your email address

I’d like to see that. You can post it here as a file attachment. See Posting a Crash Report for instructions.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Yes it's the name of the plugin (an emulation of a famous Solid State Logic mixer), not something from my application.

I join the full report, anonymized as indicated in the instructions. Initially it's a file with the .ips extension (coming from the Console app), but because it was not possible to join it, I changed the extension to .crash.

Thanks!

I think you might’ve posted the wrong crash report, because this doesn’t match the snippet you posted earlier. The new crash report’s crashing thread backtrace looks like this:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0  com.apple.CoreFoundation  … CFStringGetLength + 6 …
1  com.apple.CoreFoundation  … _CFURLCreateWithFileSystemPath + 68 …
2  com.cccccccLllll.Ssssssss …
3  com.cccccccLllll.Ssssssss …
4  com.apple.AppKit          … -[NSSavePanel didEndPanelWithReturnCode:] + 114 …
…

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Oups, sorry for that ! (once opened in a text editor, such a report becomes difficult to read...) Ok, since the same beta tester was here today, we tried again the whole sequence of operations and I have some more precise results:

Actually, the most common bug I describe in my first message don't produce any crash report. They just produce a big binary file (that seems to be an image of the bundle, I could upload it if you want) saved in the temp folder of the host application, display a first warning described in my previous message, and finally this one:

Even if the application didn't crash, we were not able to open any of these plugins with my application (practically they are unusable). I don't know if it could help, but it appears that all of these plugins are protected by iLok keys (https://www.ilok.com). Moreover they all open without problem in Logic Pro and Ableton Live.

On the other hand, the partial crash report I joined was related to an issue having some specificities: it also crashed the host application so that it produced a crash report. This second category seems related to less plugins, all related to the same manufacturer. It could be a problem of the plugin itself, however we were also able to open such plugins in Logic Pro without encountering the same issue.

I join a new report produced with a plugin of the latter type:

Actually, the most common bug I describe in my first message don't produce any crash report.

Hmmm, that’s surprisingly unhelpful. Who’s displaying that alert? The plug-in itself?

once opened in a text editor, such a report becomes difficult to read...

Yep.

One trick here is to save the file as a .ips and then do a Quick Look in the Finder. That shows the crash report in the traditional human readable format.

I join a new report produced with a plugin of the latter type:

Thanks.

it appears that all of these plugins are protected by iLok keys

Yeah, I suspected DRM was involved.

In your latest crash report I see this:

Exception Type:        EXC_BAD_ACCESS (SIGBUS)
…

VM Region Info: 0x12c3a3798 is in 0x12c399000-0x12c4c2000…
…
--->  __TEXT                      12c399000-12c4c2000    [ 1188K] r-x/rwx …

You’re getting a bus error while access the __TEXT segment, that is, the program’s code. Now consider this:

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   bx_console SSL 9000 J … 0x12c399000 + 33486
1   bx_console SSL 9000 J … 0x12c399000 + 30209
2   dyld                  … dyld4::RuntimeState::notifyObjCInit(dyld4::Loader const*) + 170
3   dyld                  … dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array&) const +…
4   dyld                  … dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const +…
5   dyld                  … dyld4::APIs::dlopen_from(char const*, int, void*) + 592
6   CoreFoundation        … _CFBundleDlfcnLoadBundle + 149

So the code seems to be doing this to itself. And this:

Error Code:      0x00000007 (invalid protections for user data write)

which suggests it’s trying to write to its own code.

One nice thing about macOS crash reports is that they contain this:

Thread 0 instruction stream:
  3b 6e 85 6c b8 31 1e 1d-a2 5f 27 23 f1 07 4a 1f  ;n.l.1..._'#..J.
  fe cc 1d f0 52 2e 04 62-11 13 8e ef 90 ab 02 6e  ....R..b.......n
  ab bf 2b 05 39 d2 0d b8-58 e6 2d 39 39 e7 5b 8f  ..+.9...X.-99.[.
  06 88 45 42 90 ef 8b 45-23 53 a0 c4 51 6e e9 61  ..EB...E#S..Qn.a
  8f f4 d0 12 b9 cc 82 7c-a9 69 c2 55 d3 29 18 58  .......|.i.U.).X
  e4 9c 2c 85 88 75 d3 90-12 ed e0 13 0a 05 e2 38  ..,..u.........8
 [c7]05 c0 24 00 00 00 00-00 00 e9 a1 20 00 00 cc  ...$........ ...	<==
  9f 5c 14 5d 9d 9f 46 c8-5a 41 cc 41 0f 6e 19 5f  .\.]..F.ZA.A.n._
  07 f8 26 6c 2a d8 c1 43-0d d6 1b 30 da 2f 34 98  ..&l*..C...0./4.
  15 a8 70 23 a2 e2 59 db-9a 30 19 ee e9 bb 0b 00  ..p#..Y..0......
  00 cc 34 07 a9 be 63 7a-09 c3 b1 fd 29 47 a7 ea  ..4...cz....)G..
  21 0a 6a 05 f5 d2 d5 75-eb 3d c0 2e 0a 38 c1 66  !.j....u.=...8.f

Disassembling the crash instruction reveals this:

  movl   $0x0, 0x24c0(%rip)

confirming the above.

DRM systems often perform stunts like this, and that’s one of the reasons why DTS doesn’t support them. There’s a fundamental tension between the goals of the DRM system and long-term binary compatibility. Unfortunately you seem to be an innocent bystander in all of this.

There are a bunch of variables in play here, including at least:

• App Sandbox

• Hardened Runtime

• macOS version

• Your app

• The DRMed plug-in

I also suspect that quarantine might play a role.

To make progress you need to tease these apart. Doing that without being able to reproduce the issue is hard, so my first recommendation is that you install macOS 12 and see if you can reproduce the issue locally. If you can, that’ll make the next steps easier. If you can’t, you’re going to have to bounce back and forth with your customer.

After that I suggest that you remove your main app from the equation. If you create a tiny test project that loads this plug-in, does it reproduce the issue? I’d start with an app that has both the App Sandbox and the hardened runtime enabled, just like your main app. If it reproduces the problem, you can then selectively disable either or both and see what you get.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Following your suggestions, I try to reproduce the context of the crash as much as possible. I started with macOS 13 because I realized that the test I'd already done on Ventura was inconclusive because I was using a free demo plugin that wasn't authorized with iLok. Fortunately, other demo versions using iLok exist, so I’ll try this way. If the bug is not reproductible on Ventura, I'll consider downgrading to macOS 12 or installing it on an external drive. All this is time consuming, but hopefully I'll have more info in a few days. If so, I'll share them here.

Anyway, thanks for your help!

I was finally able to reproduce the issue relatively quickly on Ventura. Here is a report of my tests (sorry for the length of this post):

• Installed Plugin with iLok protection : IRCAM-Trax from Flux Audio (demo version).
• OS: macOS 13.2.1,
• Hardware: iMac Retina 4K, 21.5-inch, 2019, 3,2 GHz Intel Core i7 6 cores
• Sandbox options: Outgoing Connection • Audio Input • User Selected Files: Read-write • Download, Music and Movies folders: Read-write

Test 1: (DEBUG Scheme) SANDBOX, HARDENED RUNTIME (Disable Library Validation)

• => Failed: Exact same behavior as in macOS 12 (except that the error message is presented fourth instead of twice). The logs provided by the console wasn't very helpful, they don't seem related to the plugin crash.

Test 2: (DEBUG Scheme) SANDBOX, HARDENED RUNTIME (Disable Library Validation • Disable Memory Protection • Allow DYLD Environnement Variables • Allow Unsigned Executable Memory • Allo Execution of JIT-compiled Code)

• => Failed: Exactly as the previous.

Test 3: (DEBUG Scheme)

• => Failed: After deactivating the sandbox and Hardened Runtime, no plugin wants to open although all plugins have been validated again (since the Application Support folder where the referenced plugins are stored has been changed). I realized that the Hardened Runtime's ‘Disable Library Validation’ key is required to open any Audio Unit plugin, even in a non-sandboxed application (probably except those from Apple ? I didn’t test it).

Test 4: (DEBUG Scheme) HARDENED RUNTIME (Disable Library Validation)

• => Failed: Here the iLok manager finally complained about the fact that it detected a debugger when I tried to open Trax… impressive.

Test 5: (RELEASE Scheme) HARDENED RUNTIME (Disable Library Validation)

• => Crashed: Strangely, with the release scheme, the application crashed when I tried to open Trax, I join the report below:

Test 6: (RELEASE Scheme) HARDENED RUNTIME (Disable Library Validation • Disable Memory Protection )

• => Success: Trax did open and all work as expected!

I was happy to finally find a solution, but the bad news is that this solution involves not using a Sandbox, which is not an option for me since I would like to distribute on the AppStore.

I still wondered how Logic and Garage Band do to work around this issue, so I tested again with Garage Band (on Monterey, the test was done with a licensed version of Logic Pro 10.7, which I think is sandboxed, and it had worked normally). But, finally, Garage Band 10.4.8 behave exactly the same way than my application:

First this message this message was displayed 4 times when I tried to open Trax:

Then, the error 100001 (reproduced in a previous post) was displayed.

Next, this more informative message was displayed:

Finally, you can see the content of the Garage Band’s temporary folder on this last screenshot: each error produces a file with another name and don’t overwrite the previous one. I hope this folder is cleared regularly because, depending on the size of the bundle, and how many times the user gets this error, this could consume some hard drive space unnecessarily...

So, it finally appears that there is a real conflict, at least from macOS 12, but it could earlier, between some plugins authorized via iLok and the OS (with the collection of my beta-tester, it was something like 75% of the iLok protected-plugins). The only remaining doubt is about the try we made with success on Logic Pro 10.7.9 and Monterey (but I have no prove that this version is sandboxed, the simplest explanation would be that it’s not...). Do you think there is something else I can do to find a solution ? It seems that these problems are far removed from my application and that I don't have much control over them. Thanks!

Looks like you’re making great progress here.

but I have no prove that this version is sandboxed

There are two ways to do that:

• Run the app, run Activity Monitor, find the app in the process list, and check the Sandbox column. You might need to control click on the list headers to enable that column.

• Check for the entitlement with codesign. I don’t have Logic installed, but here’s an example with GarageBand:

% codesign -d --entitlements - /Applications/GarageBand.app   
…
[Dict]
    …
	[Key] com.apple.security.app-sandbox
	[Value]
		[Bool] true
    …

Having said that, working with GarageBand seems like the acid test to me. If an audio unit fails there, it’s worth reporting that as a bug against the audio unit.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks for the tip, that was very helpful!

So here are the results of my last tests:

1. I was able to reproduce that bug with several other plugins from different manufacturers which have in common to be authorized with iLok. I don’t describe the symptoms here since they are exactly the same as in my previous post, but they affect dozens of plugins, all from the list of manufacturers coming from this beta-testing session (but it probably exist other cases).
2. I was also able to reproduce the bug with the VST or VST3 versions of the same plugins by opening them in an application hosting VST plugins loaded on the App Store (n-Track).
3. More surprisingly, I got many similar errors when opening Audacity, which is not a sandboxed application: each of these plugins triggered the same error message during validation. However, after quitting and relaunching the app, the plugins were finally available and worked normally.

In addition, the Logic Pro question revealed that version 10.7.9 was not sandboxed. Here's what the codesign command returns:

Executable=/Applications/Logic Pro X.app/Contents/MacOS/Logic Pro X

This command was executed on the Monterrey laptop, but I also loaded the last demo version on Ventura, and the version number is exactly the same. So, apparently, the most recent version of Logic Pro is still not sandboxed (which is confirmed by the fact that there is no container with its name in the user's library).

I would have deduced from all this that sandboxing was ultimately not a viable choice for a professional music or audio application, but I also found an iLok-protected plugin that maybe contradicts this idea:

My beta tester had pointed out that the GRM Tools plugins (from INA in Paris) didn’t crashed like the others, despite their iLok protection. I investigated this and, indeed, these plugins work perfectly in a sandboxed application on Ventura. The difference seems in the way they are authorized: whereas all other plugins open an iLok window during the validation process, and crash when they are instantiated, the GRM plugins require the customer to explicitly enter a validation code into the iLok manager after installing the plugin bundle and, after this step, there is no further (visible) interaction with iLok during validation or instantiation, and everything works as expected.

This finally suggests that manufacturers would have a solution for implementing Pace iLok in a way that is supported by a sandboxed application. But, on the other hand, there are so few sandboxed music apps that they’re probably not aware of this problem. We can therefore fear that this situation will never be resolved, especially if the Logic Pro team doesn’t move in the direction of sandboxing. I fully understand why they don’t, but it would be a great motivation (perhaps the only one) for manufacturers to fix their implementation of iLok...

Anyway, I now have many more info to think about the question of an alternative or additional distribution. Thanks a lot for your help!