App is signed and notarized, but still produces the "cannot check it for malicious software" error in Catalina

Problems like this can be tricky to debug. I have some suggestions on this thread.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks for the tips, but after pouring through a huge amount of logs, I have found nothing that I can recognize as a reason my app is being rejected by gatekeeper.

The best I can find is the following cryptic log messages:

syspolicyd   looking up ticket: {length = 20, bytes = 0x}, 2, 0
syspolicyd   completing lookup: {length = 20, bytes = 0x}, 3
syspolicyd   MacOS error: 3
syspolicyd   Error checking with notarization daemon: 3
syspolicyd   rule 6 applies - allow=1

(repeats ~20 times)

syspolicyd   looking up ticket: {length = 20, bytes = 0x}, 2, 0
syspolicyd   completing lookup: {length = 20, bytes = 0x}, 0
syspolicyd   rule 11 applies - allow=1
syspolicyd   Legacy override ignored for match source: Notarized Developer ID
syspolicyd   GK scan complete: /path/to/my.app, 0, 0
syspolicyd   App gets first launch prompt because responsibility: /path/to/my.app/Contents/MacOS/myapp, /path/to/my.app
syspolicyd   GK evaluateScanResult: 0, /path/to/my.app, 1, 0, 1, 0, 0, 0
syspolicyd   GK eval - was allowed: 0, show prompt: 1

How on earth am I supposed to glean anything meaningful from that?!

I'm sorry if I'm venting my frustrations, but this is the umpteenth time I've run into a show-stopping issue due to Catalina's security. And I find it absolutely insane that Apple provides no tool for checking whether Gatekeeper will allow an app to run on Catalina and, if it fails Gatekeeper's checks, providing clear information as to why it won't run. How can you expect people to develop software for your platform without that?

(And just as a tangential aside, why are these security requirements, most of all notarization, even necessary? What specific security problems that were present in Mojave are they solving, and to whose benefit? I ask because, as far as I can tell, this new security provides no real benefit to the end user, and rather causes extra headaches and work for developers and a lot of confusion and frustration for everyone.)

Sufficed to say, I will an enhancement request as you suggest in the thread you linked to.

I will also open a DTS technical support incident since I clearly can't solve this problem on my own. I only hope that the result of that conversation will be information that will allow me to solve this sort of issue on my own in the future. (Basically, I'm looking to learn to fish, not just to be given a fish.)

I've just built an app, signed it with my Developer ID certificate, and had it successfully notarized. However, when I download a zipped copy of the app in macOS 10.15.5 and try to run it, I get the "“XYZ” can’t be opened because Apple cannot check it for malicious software" error message.

I know this is nit-picky, but that isn't a successful notarization. A successful notarization is when you don't get that message.

There are a number of things that have to happen successfully to have successful notarization. You need a correct bundle. You need embedded frameworks/dylibs that are properly signed and/or entitled. You need need the full dynamic loading layout to be in good shape. There are a number of popular, but poor practices published on the internet, flaky 3rd party libraries, and flaky 3rd party frameworks that contribute to failed notarization. Obviously I don't know which of those are afflicting you, but I know it is one of them, it always is.

There are a few things you can do to help debug these problems:

1) Don't use poor signing practices. Don't use "--deep" and don't use "--force". I'm sorry, but the internet is wrong on this, as it often is.

2) Don't use flaky 3rd party frameworks and libraries. Yes, I'm talking directly to you, Qt. But I'm sure there are many others.

3) Use sane bundle layouts. Look at any 3rd party, properly notarized app on your machine. Look at how the app bundle and frameworks are structured. Copy that structure. In many cases, the actual fault is not with the bundle structure per se, but the structure is a big Red Flag. It means you are improvising and you going to have problems. Maybe if you tweak that one entitlement, you'll get it to work today, but it will be broken again next month.

4) Make sure your dynamic libraries are properly loaded. Use "otool -L" to check this. This is one of those "hidden" steps that the command-line verification tools don't check. In some cases, the app will work fine when you test it but then fail the download test, the "true" notarization. This is likely because your development build is using one set of dynamic libraries visible to your Xcode build or your account, but the release version is trying to do it in release mode, and failing. You can easily test this by running the app inside a VM. In many cases, you don't even need to bother notarizing to test this. There may be a note in the Console log when your app fails to load. Make sure to have Console running, then try to launch your downloaded app, then quickly check for messages in Console. Be careful with Console. It is a ***** that will bring your Mac down to its knees and leave it a smouldering pile of ashes.

why are these security requirements, most of all notarization, even necessary? What specific security problems that were present in Mojave are they solving, and to whose benefit?

Malware? Apple doesn't like to say it, but fraud and malware are HUGE. It isn't the security problems that are present, it is the security problems that aren't there. The reason you aren't aware of this problem is due to Apple's efforts to keep it at bay. Apple is doing a very good job of that. But, sometimes, 3rd party developer doing funky things with dynamic libraries are paying the cost. Apple's efforts keep most malware and fraud efforts non-viable. Smart fraudsters don't even bother. A few clueless ones try, but they are easy to deal with. But Apple's can get complacent because there is an entire industry dedicated to breaking Apple's security.

What I don't understand is that if any of these problems are present in an app bundle, why does Apple's notary service still report success and allow me to staple a ticket to the app? I've had many failed notarization attempts before, and it provides me with a report of everything that's wrong in the app, including frameworks, dylibs, or other binaries that are improperly signed, files being in the wrong place in the bundle, executables not being built with hardened runtime, and so on.

I know this is nit-picky, but that isn't a successful notarization. A successful notarization is when you don't get that message.

I think the distinction is between the app being accepted by Apple's notary service and them providing a ticket to staple, and the app passing Gatekeeper's security checks. The former is necessary for the latter, but the latter apparently has more requirements beyond just notarization. Unfortunately it also doesn't tell me anything about why it's failing to allow my app to run!

1) Don't use poor signing practices. Don't use "--deep" and don't use "--force". I'm sorry, but the internet is wrong on this, as it often is.

I have been using this since, at up until now, it had been working fine. Is the alternative you recommend to individually sign all bundles, executables and dylibs from deepest to shallowest in the bundle?

Also, if there is something wrong with these two options, I would hope Apple would either get around to fixing them or remove them if they don't work and aren't supported.

2) Don't use flaky 3rd party frameworks and libraries. Yes, I'm talking directly to you, Qt. But I'm sure there are many others.

Too late for that -- this entire app I'm trying to get notarized is built using Qt 5.9.9. However I had no problem getting it notarized and running in macOS 10.15 until now. Very little of consequence has changed between my prior and current attempts!

3) Use sane bundle layouts.

I've already had to restructure my app bundle in order to get everything in proper order so that Apple's notary service would accept it. But as near as I can tell, everything is assembled correctly now. If there is an issue, I thought Apple's notary service would reject the app and let me know.

4) Make sure your dynamic libraries are properly loaded.

Everything is linked using absolute paths or @rpath. Is @rpath still allowed? I've had Apple's notary service reject an app because of improper linkage, so I thought that if it's accepted that'll be the end of it.

Malware?

Sure, I figure that's the ostensible reason. But for one, I don't see how this provides more protection than what Mojave already had. There was already a system in place to catch known malware and prevent it from running, plus developer certificates can be revoked by Apple if a dev gets caught committing abuses. And unsigned apps were already rejected outrightly by Gatekeeper.

And more importantly, I've yet to see any compelling explanation why this entire notarization system works better than, say, some kind of client-side process that scans executables to try and detect malicious code the first time it's executed. That's been a successful method of stopping malware on every other operating system. It also seems to have a lot of important advantages:

- Malware definitions can be updated frequently. If a novel piece of malware manages to slip through Apple's notarization checks and is notarized, it'll still appear to be legitimate, whereas as-needed scanner could detect it once the malware definitions are updated.

- It avoids all of the aggravating and frustrating work developers have had to go through getting their software notarized

- It provides a better user experience because it provides just as much protection while still allowing more apps to run.

- It allows the platform to remain more open.

What I don't understand is that if any of these problems are present in an app bundle, why does Apple's notary service still report success and allow me to staple a ticket to the app?

Because it is likely a runtime failure. You didn't notice because you were running in whatever development environment that Qt provides. But Gatekeeper does additional runtime dylib checks.

Unfortunately it also doesn't tell me anything about why it's failing to allow my app to run!

Have you looked in Console? One of the times I responded to a question virtually identical to yours, I found a dylib loading problem that was causing notarization to fail. There aren't any easy-to-use error messages for this because it only affects people who aren't using Xcode. Xcode just works. But if you choose to roll your own, you have to be willing to debug the errors on your own too.

I have been using this since, at up until now, it had been working fine.

This is what I mean when I say, "you'll get it to work today, but it will be broken again next month."

Is the alternative you recommend to individually sign all bundles, executables and dylibs from deepest to shallowest in the bundle?

The "alternative" is to use Xcode which does it all correctly. Failing that, you could look to see how any other app using Xcode is built and copy that.

However I had no problem getting it notarized and running in macOS 10.15 until now.

See my note above.

Very little of consequence has changed between my prior and current attempts!

There have been tons of changes. In my previous message, I said, "there is an entire industry dedicated to breaking Apple's security." I wasn't joking. This is how it works. Apple has to patch these bugs or their authors will release them. It is legalized extortion. This is where you see the ramifications. One of those exploits used the same slightly lax dylib loading scheme that you depended on. That bug is now fixed, so your app is broken.

Is @rpath still allowed?

@rpath is the preferred method. However @rpath is dynamic. Qt libraries, especially if you build them yourself, may have funky rpath values that specifically cause this problem.

And more importantly, I've yet to see any compelling explanation why this entire notarization system works better than, say, some kind of client-side process that scans executables to try and detect malicious code the first time it's executed. That's been a successful method of stopping malware on every other operating system.

Well, no. In fact is has been the opposite. Apple's security platform has been more successful than any client-side approach. And it certainly provides a better user experience.

And don't forget that I also mentioned fraud. Malware is what everyone is scared of and is what drives the Apple security and clickbait industry. But it is largely just fearmongering. Apple's security infrastructure works very well and there is very little risk. Nobody talks about fraud. Fraud is orders of magnitude bigger than malware. Most of these "security exploits" are also useful for fraud. But in this case, it is developers being defrauded. My piracy rate is around 20% for direct sales. (Mac App Store appears to be 0%.) And I do a lot of DRM. It is a free app, with in-app purchase. But in order to run the pirated version, you have to disable Apple security. I'm nobody. I can't imagine how bad it is for big developers.

- Malware definitions can be updated frequently. If a novel piece of malware manages to slip through Apple's notarization checks and is notarized, it'll still appear to be legitimate, whereas as-needed scanner could detect it once the malware definitions are updated.

It doesn't work that way. Apple has multiple levels of security. For all practical purposes, users have to install malware themselves by overriding Apple security.

- It avoids all of the aggravating and frustrating work developers have had to go through getting their software notarized

Notarization is fall-off-a-log easy, for everyone who uses Xcode, properly signs, and avoids problematic frameworks.

- It provides a better user experience because it provides just as much protection while still allowing more apps to run.

Nope.

- It allows the platform to remain more open.

Wrong platform, Corrigan. That doesn't matter here.

> Have you looked in Console?

Yes, my first post that you replied to contained excerpts from all of the relevant parts of Console's output that I could find. Unfortunately it didn't give me any clues.

> The "alternative" is to use Xcode which does it all correctly.

> Notarization is fall-off-a-log easy, for everyone who uses Xcode, properly signs, and avoids problematic frameworks.

I hope you can understand and appreciate that it doesn't make sense to use something like Xcode for every software project on macOS. Xcode is highly opinionated and platform-specific. In my case I'm using Qt's toolchain out of necessity for my paid work. In my experience, it's not always practical or possible to do things the way Apple prescribes. It simply doesn't cover all possible use cases or project needs.

Besides, Apple provides command-line tools that are supposed to allow developers to use other development tools outside of Xcode. I'm using those tools as documented by Apple and they're failing to produce software that works in macOS 10.15.5 while erroneously indicating that they were successful. I consider that a bug in Apple's tools, not a flaw in my development toolchain.

> One of those exploits used the same slightly lax dylib loading scheme that you depended on. That bug is now fixed, so your app is broken.

It's still unclear if the problem with my app is about dylib loading. At least there's no clear output in Console about an issue with dynamic libraries. And all of the dylibs required by Qt are signed, notarized, and in the proper place in the app's bundle.

> Well, no. In fact is has been the opposite. Apple's security platform has been more successful than any client-side approach. And it certainly provides a better user experience.

I agree that Apple has been a lot more successful than Windows. (I'm not really going to compare to Linux since by and large Linux can be configured however the user wants and therefore security is up to the user.) However I'm not suggesting a client-side approach replace anything other than notarization. All the rest of it could stay as is. So what I'm saying is that there doesn't seem to be any reason why the checks that Apple's notary service performs couldn't just happen client-side, relieving developers of the burden of having to notarize while allowing for more malware to be caught. I'm sure malicious parties can and already have figured out how to get malicious software notarized through obfuscation and exploiting current security flaws in the system, and those notarized apps will continue to appear to be valid to Gatekeeper. That is, of course, unless macOS receives updates that allows it to catch those apps. But in that case, why not just do the notary service's job client-side anyway if it requires constantly updating the system with new methods for detecting malware like any other anti-malware process?

> And don't forget that I also mentioned fraud.

Notarization hasn't stopped piracy at all. The only thing that I'm aware of being stopped is pirated apps that include server-side features like iCloud integration where an app needs to be properly signed and provisioned by a specific developer. And in this case the security mechanism that's preventing use of the pirated feature is codesigning, not notarization.

> Wrong platform, Corrigan. That doesn't matter here.

It most certainly does. macOS is not iOS. It is (and hopefully will continue to be) a general purpose computer operating system that's meant to be used for however its users need it. In the case of developers, power users, and other advanced users, it might require doing non-standard, even unorthodox, things that the system isn't originally designed to do. The more restrictions that Apple applies to what software is allowed to run and do, the more these use cases get locked out and the less useful the platform will become. This is just my opinion but I think it would be very short-sighted of Apple to continue along these lines.

Yes, my first post that you replied to contained excerpts from all of the relevant parts of Console's output that I could find. Unfortunately it didn't give me any clues.

It's just a suggestion. I tried it once on a problem virtually identicaly to what you describe in this same forum. I found the problem right away in Console. Did I mention how messed up Console is. I'm used to dealing with it. If you aren't familiar with it, you might miss that one significant line in the 10,000 irrelevent ones generated during that 6 second test.

I'm using those tools as documented by Apple and they're failing to produce software that works in macOS 10.15.5

Someone should come up with an aphorism about a worker blaming his tools...

You know Xcode works. If you don't, then I'm telling you it does. You can build a demo app and notarize it yourself to confirm. For extra fidelity, embed some random framework. Say, why not Qt? You don't have to do anything fancy. There has go to be some simple function to print out the version or something. Maybe even give your app the same bundle ID as your real app. Xcode gives you a comprehensive log file of everything it does. It's not like Console. It's useable. Compare that to what your own toolchain is doing. Change anything that is different than what Xcode does. Compare the structure of your demo app with your real app. Change the real app to match. Check the dylib paths and the rpath in the executable and compare those, fixing anything that is different.

It's still unclear if the problem with my app is about dylib loading. At least there's no clear output in Console about an issue with dynamic libraries. And all of the dylibs required by Qt are signed, notarized, and in the proper place in the app's bundle.

Hey, it's an idea. That's something isn't it. You have a better idea? Great! Run with it! It has been the right answer in 93% of the other cases of people with crazy Qt projects that won't notarize. Have I mentioned that this is ALWAYS Qt? Because if I haven't pointed that out yet, you should really be aware of it.

burden of having to notarize

It's not a burden. It's drop-dead, fall-of-a-log, do-it-in-my-sleep easy.

Notarization hasn't stopped piracy at all.

How do you know? And what do you care? Notarization is the single easiest part of the entire process. There is absolutely nothing about making an app that is easier than Notarization. Not a single thing!

macOS is not iOS.

macOS has been nothing but ports of iOS for years.

I think it would be very short-sighted of Apple to continue along these lines.

You better go back into quarantine because you sure aren't going to like the next few weeks.

> You know Xcode works. If you don't, then I'm telling you it does.

I know Xcode works, but it only works for the things it works for. If you're working on a cross-platform Qt project, then the best choice is to use Qt's IDE. It is possible to build a Qt project using Xcode of course, but you'll either have to rely on a qmake generated Xcode project which comes with its own can of worms, or manually keep your Xcode project in sync with your Qt project. There's no perfect solution here. But I hope you can understand that it's not always as simple as "just use Xcode".

(And I'll just briefly mention the actual experience of using Xcode versus other IDEs. Xcode is slow, kludgey, buggy, and difficult if not impossible to customize in important ways in comparison to other IDEs, which makes it very undesirable to me. I'm very opinionated about the workflow, UI arrangement and auxilery tools (like linters and code formatters) that I want to use and when writing code, and Xcode constantly puts obstacles in the way of that. Its general modus operandi is to do things Apple's way, or not do them at all. It's my least favorite IDE, except for maybe Visual Studio.)

> It's not a burden. It's drop-dead, fall-of-a-log, do-it-in-my-sleep easy.

We'll just have to agree to disagree here. I mean, I do get it. If you're just building a typical macOS application in Xcode and using Xcode's automatic workflow for building, signing, provisioning, and notarizing a release, then it makes it very easy. I've built many such applications entirely within Xcode and notarization went smoothly. But it only applies to, as I said, typical macOS applications.

For my work I've lost many days bashing my head against notarization, especially because my company releases custom-built drivers and software that's generated automatically based on hardware profiles. It's not impossible to add notarization to that, but it was definitely difficult. Yes, we're building software in non-standard ways, but we do that because we have to. There's just no realistic way to run that kind of pipeline for our mac software only using Xcode.

> You better go back into quarantine because you sure aren't going to like the next few weeks.

Yes, I suspect I won't! 😝

After opening a TSI, eskimo was able to help me sort out what I was doing wrong. (Thanks!)

The issue was that the scripts I wrote for zipping an app to send to Apple's notary service had a fatal (and apparently common) bug. I was using ditto like so to zip the app:

ditto -ck -rsrc --sequesterRsrc /path/to/myApplication.app

What's missing is the --keepParent option. It should have been this:

ditto -ck -rsrc --sequesterRsrc --keepParent /path/to/myApplication.app

Without --keepParent the topmost directory in the zip will be "Contents" rather than "myApplication.app", and the ticket won't contain cdhashes for anything except the app's main executable.

Hopefully this thread will allow other people to avoid this pitfall in the future!

And I do sincerely hope that Apple will improve their command line tools and the console output from Gatekeeper to make this sort of issue much easier to debug. If Gatekeeper's output had said something to the effect of "xyz framework in myApplication.app is not included in its ticket" or if xcrun's stapler command issued a warning saying the ticket didn't include any of its frameworks or dylibs then it's much more likely I could have figured this out on my own.

I followed everything and and successfully stable my app but when I run it, it says:

"You do not have permission to open the application "myapp"

Cracking me head over many days and still can't get anywhere..

There are two potential issues here:

• Gatekeeper

• Other trusted execution problems

The “You do not have permission to open the application” error is the latter, not the former. It means that the Finder has asked the system to start your app but something went wrong when it tried to do so. For example:

• On this thread it was because the app was modifying itself, and thus breaking the seal on its own code signature.

• On this thread it was because the app was built for arm64e, which we don’t support for third-party user space code.

As to what’s causing your specific problem, it’s hard to say. My advice is that you start a new thread with some details about your app. Specifically:

• When it fails in this way, does it generate a crash report? If so, include that in your new thread.

• What do you see if you run the executable directly from Terminal, that is:

  % MyApp.app/Contents/MacOS/MyApp
  

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"