Manual codesign / notarization and entitlements

Hi all,


We're trying to package a new version of Multipass, a small cross-platform Linux VM manager. It's been working fine until recently, when the notarization service started erroring out on binaries unsigned, or those that don't have the hardened runtime enabled.


We're using CPack to create a custom installer, so we have to do all the signing and notarization manually.


Unfortunately the hypervisor we use (hyperkit) fails when ran with hardening:

CODE SIGNING: 31277[hyperkit] vm_map_protect can't have both write and exec at the same time


While we investigate that problem, we wanted to add the appropriate entitlements to the signature, please tell me if there's something wrong with this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>application-identifier</key>
    <string>com.canonical.multipass.hyperkit</string>
    <key>com.apple.security.cs.disable-executable-page-protection</key>
    <true/>
</dict>
</plist>


Unfortunately even though notarization completes on the package, the binary fails with:

mac_vnode_check_signature: /Library/Application Support/com.canonical.multipass/bin/hyperkit: code signature validation failed fatally: When validating /Library/Application Support/com.canonical.multipass/bin/hyperkit:
  Code has restricted entitlements, but the validation of its code signature failed.
Unsatisfied Entitlements: proc 35751: load code signature error 4 for file "hyperkit"


Following TN2318 I can see that the signature verification fails if I use the "Developer ID" identity, which notarization seems to require (?):

$ codesign --verify -vvvv -R='anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.1] exists and (certificate leaf[field.1.2.840.113635.100.6.1.2] exists or certificate leaf[field.1.2.840.113635.100.6.1.4] exists)' "/Library/Application Support/com.canonical.multipass/bin/hyperkit"
/Library/Application Support/com.canonical.multipass/bin/hyperkit: valid on disk
/Library/Application Support/com.canonical.multipass/bin/hyperkit: satisfies its Designated Requirement
test-requirement: code failed to satisfy specified code requirement(s)


If I use an "Apple Development" identity, the check completes:

$ codesign --verify -vvvv -R='anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.1] exists and (certificate leaf[field.1.2.840.113635.100.6.1.2] exists or certificate leaf[field.1.2.840.113635.100.6.1.4] exists)' "/Library/Application Support/com.canonical.multipass/bin/hyperkit"
/Library/Application Support/com.canonical.multipass/bin/hyperkit: valid on disk
/Library/Application Support/com.canonical.multipass/bin/hyperkit: satisfies its Designated Requirement
/Library/Application Support/com.canonical.multipass/bin/hyperkit: explicit requirement satisfied


But even then, the above "signature failed" failure occurs. Not to mention that files signed with that identity get rejected during notarization…


Any pointers on what are we doing wrong?


Thanks a bunch!

Accepted Reply

Notarisation requires the hardened runtime. The hardened runtime enables all sorts of extra security checks by default. However, most (maybe even all?) of those can be disabled with hardened runtime exception entitlements. These are not special entitlements (that is, entitlements that must be granted by Apple), nor are they restricted entitlements (that is, entitlements that must be whitelisted by a provisioning profile), but are completely open. Any Developer ID code can use these entitlements without restrictions.

The problem with your original attempt is that you’re using a restricted entitlement (

application-identifier
) without a profile. However, the other entitlement you used (
com.apple.security.cs.disable-executable-page-protection
) is not restricted, and you should be able to sign your binary with it.

As to whether that’ll fix your actual problem, it’s hard to say without knowing more about how your code is using

vm_map_protect
.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Replies

If anyone wants to take a look, the notarized and stapled installer can be downloaded here: multipass-1.1.0-dev.600+g43acabe8.mac-Darwin.pkg.

I suggest bundling all of this up into a proper app and resubmitting it. Clearly, Apple knows what apps are supposed to look like and they've designed the system to work with recognizable apps. If you feed it something else, who knows what you'll get back.


As an added bonus, if you bundle this up into an app, you don't need an installer package either. Or an uninstaller script. Or root.


Edit: I took another look at your installer scripts. I see the launch daemon now. See if you can replace that with a login item. You can still do it all in a single app (without having to construct some quasi-app in /Applications) but you need an uninstaller, and maybe root, with that launch daemon. This app would be much nicer without it. You could probably even get it into the Mac App Store.


Edit: Sorry. I took yet another look at your installer script. I'm not sure if you can get it into the Mac App Store with Qt and those "gnu" files. I'm not a lawyer so I'm not going any further on that. But I stand by my claim that you can wrap this up into a more recognizable app and it should work fine then. And I think it would be very easy to strip out the gnu stuff and get it into the Mac App Store.

Hi john, thanks for your suggestions.


As you realized yourself, it's not a trivial application, partly because it's cross-platform, and we'd rather not have to maintain a completely separate build / packaging story for each platform.


Either way, we'd like to be able to get it working before we go on a "refactoring" spree, "should work" is a bit too little certainty…


I hope that, at least a short-term, solution would be to fix the signature / entitlements, but I've hit a wall as to where to look to understand what's going wrong.

While we investigate that problem, we wanted to add the appropriate entitlements to the signature, please tell me if there's something wrong with this

There’s something wrong with this (-:

The

application-identifier
entitlement is incorrect on the Mac; it should be
com.apple.application-identifier
[1]. However, that’s only part of that problem. Those entitlements are restricted, meaning that they must be whitelisted by a provisioning profile. My guess is that this is not what you want [2], in which case you should simply remove it.

It’s possible that there are other problems in play here, but I’m going to recommend that you fix this first and then let us know how that panned out.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

[1] The exception being Mac Catalyst apps, which have both!

[2] Generally, you only need a profile if you want to use some other restricted entitlement, and

com.apple.application-identifier
comes along for the ride (it’s part of the mechanics that bind your code to your profile).

Hi eskimo,


Yes, this was my (failed, ill-advised) attempt to avoid the hardened runtime from killing `hyperkit` because of:

CODE SIGNING: 31277[hyperkit] vm_map_protect can't have both write and exec at the same time 


Is there some other way I can include an exemption for that binary? Or is notarization only currently possible with the hardened runtime enforced on all binaries?

Notarisation requires the hardened runtime. The hardened runtime enables all sorts of extra security checks by default. However, most (maybe even all?) of those can be disabled with hardened runtime exception entitlements. These are not special entitlements (that is, entitlements that must be granted by Apple), nor are they restricted entitlements (that is, entitlements that must be whitelisted by a provisioning profile), but are completely open. Any Developer ID code can use these entitlements without restrictions.

The problem with your original attempt is that you’re using a restricted entitlement (

application-identifier
) without a profile. However, the other entitlement you used (
com.apple.security.cs.disable-executable-page-protection
) is not restricted, and you should be able to sign your binary with it.

As to whether that’ll fix your actual problem, it’s hard to say without knowing more about how your code is using

vm_map_protect
.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

As you realized yourself, it's not a trivial application


Well. I didn't say that. 😉


In fact, I thought it would be pretty trivial to just reorganize your /Library/Application Support/com.canonical/multipass directory into a proper app. I had decided to swear off these notarization questions entirely. But then I saw some activity in this thread and thought this would be an easy one to solve, and make an example out of you.


Well, I was right, and wrong. I was able to succcessfully repackage your app into a somewhat-real app bundle, sign it, and notarize it. It doesn't run, of course, but that's out of scope. It wasn't difficult to fix it. I did it all from the command line. But it was more difficult that I had expected.


What you are attempting to notarize isn't anywhere close. You can construct an app bundle manually. I've done it. But you have to do it correctly. There are many examples of working app bundles on your Mac. Copy one of them. Make sure you have an application bundle ID. I'm sure you have examples of other apps that have embedded dylibs. Look at how those are constructed. I'm sure you have many apps that have embedded frameworks. What does a framework look like? I complain about Qt on a regular basis. But Qt does provide developers with working frameworks. Use them! Don't try to cobble something together on your own and call it a framework. That's not a framework.


Once you get past those basic problems, you'll want to double-check what you are doing with rpaths in the executable and inside the "frameworks". I tried to hack up the rpaths in the executable, but it turns out those absolute paths that only exist on your travis build machine are scattered throughout the Qt dylibs too. I'm tired on hacking on it now. It won't run anyway because of however the Qt plugin archtecture works. You can't blame this on cross-platform issues. You can fix all of this on the command line. I'm sure Linux has a scriptable shell you can use.


You have done nothing to convince me that notarization is anything other than fall-of-a-log-easy. All you need to do is start with a valid executable. Apple seems to have been very lax about what constitutes a valid executable in the past. They seem to have tried hard to make sure that any old executable will run. Well, the modern notarization flow is more strict than that. Your app bundle needs to look like an app bundle. Your frameworks have to be real frameworks. Your info.plist has to be valid. Your dynamic libraries have to be properly linked. That all seems like a very reasonable starting point.

Thank you eskimo!


Just removing the wrong entitlement did the trick.