Catalina: how do I determine why Gatekeeper is rejecting a signed executable?

We have a multi-platform suite of command-line executables and libraries that we ported to Mac; I'm new to Mac development. The file layout was

/Applications/

  • (company folder)/
    • (our UI).app
    • (product name)/
      • bin/
        • ...executalbes...
      • lib/
        • ...dylibs...
      • (other stuff)/...


This was shipped in a DMG that was codesigned, as was the app. This worked OK until Catalina.


Now on Catalina, we have codesigned all the executables, dylibs, apps (including ones nested in the top-level app's framework), frameworks, and the DMG itself. When we notarize it the resulting JSON log lists no issues. However, when I run any of our executables that depends on one of our dylibs I get a pop-up telling me the "developer cannot be identified". Even though it has been signed and notarized OK. Running codesign with -dvvv option includes the following:

  • SHA-256 hash choice
  • list of Authority entries terminating in Apple Root CA
  • TeamIdentifier entry
  • Timestamp
  • Runtime: 10.13.0

Question How can I fix this, or at least get Gatekeeper to tell me why it's not accepting this file? Maybe a log, or an equivalent of spctl --assess for files rather than apps?


Observations

    • This only happens when
      • the OS is Catalina
      • it's under the /Applications folder
      • outside of /Applications (e.g. in a folder on desktop) it only happens sometimes (and sometimes it claims a dylib can't be loaded on first attempt, then succeeds if tried a moment later)
      • the executable depends on one or more of our dylibs (standalone ones run OK)
      • the executable has the com.apple.quarantine xattr set
        • I've tried to mix'n'match between clean and downloaded (i.e. quarantine-xattr'd files) and the problem only arises if the executable is quarantined; it doesn't care if a non-quarantined executable loads a quaratined dylib
    • The signing operation was done via codesign with args "--deep --strict --timestamp --options runtime", and then verified
      • I've since updated this to include some of the Hardened Runtime entitlements to fix another build issue, but it hasn't helped with this one
    • The executables depend on the dylibs via @rpath (as reported by otool -L)
    • Result of "spctl --assess -t open --context context:primary-signature -v" on the DMG
      • accepted
      • source=Notarized Developer ID
    • Result of "spctl --assess --context context:primary-signature -v" on the app
      • accepted
      • source=Notarized Developer ID
    • Result of running it on the various nested apps in the app's Contents/Framework/nwjs Framework.framework/Helpers/ folder
      • rejected (the code is valid but does not seem to be an app)
      • (I'm not sure this is relevant to the problem at hand, but I just ran into it.)

Replies

That is a very unusual app design. Maybe you can get it to work one day. I don't know. It does seem odd to devote a lot of effort just so you can maintain an unusual layout like that.


I suggest looking at some other apps to see how they do it. Look at pretty much any app other than crazy open-source, cross platform apps. That will just get you off into another wrong direction.


The layout should look like this:

/Applications

YourApp.app

  • Contents
    • MacOS
      • YourApp
    • Resources
      • (other stuff)
    • Frameworks
      • ...dylibs...
    • Helpers
      • ...executables...


Divert from this at your peril.

How can I … get Gatekeeper to tell me why it's not accepting this file?

Unfortunately that is quite tricky:

  • In most cases where Gatekeeper denied execution, there is some evidence in the system log. Alas, it’s not always easy to spot. Some key terms to look for are:

    • gk
      , for Gatekeeper
    • xprotect
      , an internal name for a Gatekeeper subsystem
    • syspolicyd
      , see its man page
    • cmd
      , for Mach-O load command oddities
  • Gatekeeper caches its assessments, and if you ‘hit’ that cache then you may not see anything interesting in the log (because the code that logged the interesting stuff isn’t run). I generally test this stuff in a VM:

    1. I start with a fresh VM that’s never seen my app.

    2. I set up the VM exactly how I need it set up (install the app, prime

      sudo
      , and so on).
    3. I take a snapshot.

    4. I run the program.

    5. I collect my logs.

    6. If I need to run it again, I restore from the snapshot.

  • With regards collecting logs, you can use a sysdiagnose for that but I generally find it faster to run

    log collect
    . I then export the log to my real Mac so that I can analyse it offline.
  • In many cases critical log entries have private info redacted. I disable that system-wide using a configuration profile. See the discussion of

    Enable-Private-Data
    property in SystemLogging.

    IMPORTANT This is on my VM, of course. You really don’t want to do that on any Mac you care about.

Two final things:

  • If you get stuck, open a DTS tech support incident and I’ll try to help out there. That’ll allow me to dedicate more time to your issue.

  • If you’d like Gatekeeper to provide better facilities for debugging problems like this, file an enhancement request along those lines. Please post your bug number, just for the record.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I've opened a DTS support incident, ref 100947219881

I'm having the same issue, and I'm using an identical process to sign/notarize as another executable within my organization, however I have dylibs and that's the only difference. Everything is successful, but Gatekeeper seems to be blocking anything referencing a dylib. All the dylibs are codesigned and validated to be as such.


Also, regardless of this, 'executing' a command line from double clicking via the DMG mount still flags security warning, but runs fine from command line (when not referencing dylib), that's also pretty annoying.


Can I ask if any date on an updated Catalina with a fix for this stuff?

I’ve responded on the other thread you posted to.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Earlier I wrote:

I generally test this stuff in a VM

For more details on this, see Testing a Notarised Product.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"