8 Replies
      Latest reply on Feb 17, 2020 1:05 AM by Saviq
      Saviq Level 1 Level 1 (0 points)

        Hi all,

         

        We're trying to package a new version of Multipass, a small cross-platform Linux VM manager. It's been working fine until recently, when the notarization service started erroring out on binaries unsigned, or those that don't have the hardened runtime enabled.

         

        We're using CPack to create a custom installer, so we have to do all the signing and notarization manually.

         

        Unfortunately the hypervisor we use (hyperkit) fails when ran with hardening:

        CODE SIGNING: 31277[hyperkit] vm_map_protect can't have both write and exec at the same time

         

        While we investigate that problem, we wanted to add the appropriate entitlements to the signature, please tell me if there's something wrong with this:

        <?xml version="1.0" encoding="UTF-8"?>
        <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
        <plist version="1.0">
        <dict>
            <key>application-identifier</key>
            <string>com.canonical.multipass.hyperkit</string>
            <key>com.apple.security.cs.disable-executable-page-protection</key>
            <true/>
        </dict>
        </plist>
        

         

        Unfortunately even though notarization completes on the package, the binary fails with:

        mac_vnode_check_signature: /Library/Application Support/com.canonical.multipass/bin/hyperkit: code signature validation failed fatally: When validating /Library/Application Support/com.canonical.multipass/bin/hyperkit:
          Code has restricted entitlements, but the validation of its code signature failed.
        Unsatisfied Entitlements: proc 35751: load code signature error 4 for file "hyperkit"

         

        Following TN2318 I can see that the signature verification fails if I use the "Developer ID" identity, which notarization seems to require (?):

        $ codesign --verify -vvvv -R='anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.1] exists and (certificate leaf[field.1.2.840.113635.100.6.1.2] exists or certificate leaf[field.1.2.840.113635.100.6.1.4] exists)' "/Library/Application Support/com.canonical.multipass/bin/hyperkit"
        /Library/Application Support/com.canonical.multipass/bin/hyperkit: valid on disk
        /Library/Application Support/com.canonical.multipass/bin/hyperkit: satisfies its Designated Requirement
        test-requirement: code failed to satisfy specified code requirement(s)
        

         

        If I use an "Apple Development" identity, the check completes:

        $ codesign --verify -vvvv -R='anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.1] exists and (certificate leaf[field.1.2.840.113635.100.6.1.2] exists or certificate leaf[field.1.2.840.113635.100.6.1.4] exists)' "/Library/Application Support/com.canonical.multipass/bin/hyperkit"
        /Library/Application Support/com.canonical.multipass/bin/hyperkit: valid on disk
        /Library/Application Support/com.canonical.multipass/bin/hyperkit: satisfies its Designated Requirement
        /Library/Application Support/com.canonical.multipass/bin/hyperkit: explicit requirement satisfied
        

         

        But even then, the above "signature failed" failure occurs. Not to mention that files signed with that identity get rejected during notarization…

         

        Any pointers on what are we doing wrong?

         

        Thanks a bunch!

        • Re: Manual codesign / notarization and entitlements
          Saviq Level 1 Level 1 (0 points)

          If anyone wants to take a look, the notarized and stapled installer can be downloaded here: multipass-1.1.0-dev.600+g43acabe8.mac-Darwin.pkg.

          • Re: Manual codesign / notarization and entitlements
            john daniel Level 4 Level 4 (580 points)

            I suggest bundling all of this up into a proper app and resubmitting it. Clearly, Apple knows what apps are supposed to look like and they've designed the system to work with recognizable apps. If you feed it something else, who knows what you'll get back.

             

            As an added bonus, if you bundle this up into an app, you don't need an installer package either. Or an uninstaller script. Or root.

             

            Edit: I took another look at your installer scripts. I see the launch daemon now. See if you can replace that with a login item. You can still do it all in a single app (without having to construct some quasi-app in /Applications) but you need an uninstaller, and maybe root, with that launch daemon. This app would be much nicer without it. You could probably even get it into the Mac App Store.

             

            Edit: Sorry. I took yet another look at your installer script. I'm not sure if you can get it into the Mac App Store with Qt and those "gnu" files. I'm not a lawyer so I'm not going any further on that. But I stand by my claim that you can wrap this up into a more recognizable app and it should work fine then. And I think it would be very easy to strip out the gnu stuff and get it into the Mac App Store.

              • Re: Manual codesign / notarization and entitlements
                Saviq Level 1 Level 1 (0 points)

                Hi john, thanks for your suggestions.

                 

                As you realized yourself, it's not a trivial application, partly because it's cross-platform, and we'd rather not have to maintain a completely separate build / packaging story for each platform.

                 

                Either way, we'd like to be able to get it working before we go on a "refactoring" spree, "should work" is a bit too little certainty…

                 

                I hope that, at least a short-term, solution would be to fix the signature / entitlements, but I've hit a wall as to where to look to understand what's going wrong.

                  • Re: Manual codesign / notarization and entitlements
                    john daniel Level 4 Level 4 (580 points)

                    As you realized yourself, it's not a trivial application

                     

                    Well. I didn't say that.

                     

                    In fact, I thought it would be pretty trivial to just reorganize your /Library/Application Support/com.canonical/multipass directory into a proper app. I had decided to swear off these notarization questions entirely. But then I saw some activity in this thread and thought this would be an easy one to solve, and make an example out of you.

                     

                    Well, I was right, and wrong. I was able to succcessfully repackage your app into a somewhat-real app bundle, sign it, and notarize it. It doesn't run, of course, but that's out of scope. It wasn't difficult to fix it. I did it all from the command line. But it was more difficult that I had expected.

                     

                    What you are attempting to notarize isn't anywhere close. You can construct an app bundle manually. I've done it. But you have to do it correctly. There are many examples of working app bundles on your Mac. Copy one of them. Make sure you have an application bundle ID. I'm sure you have examples of other apps that have embedded dylibs. Look at how those are constructed. I'm sure you have many apps that have embedded frameworks. What does a framework look like? I complain about Qt on a regular basis. But Qt does provide developers with working frameworks. Use them! Don't try to cobble something together on your own and call it a framework. That's not a framework.

                     

                    Once you get past those basic problems, you'll want to double-check what you are doing with rpaths in the executable and inside the "frameworks". I tried to hack up the rpaths in the executable, but it turns out those absolute paths that only exist on your travis build machine are scattered throughout the Qt dylibs too. I'm tired on hacking on it now. It won't run anyway because of however the Qt plugin archtecture works. You can't blame this on cross-platform issues. You can fix all of this on the command line. I'm sure Linux has a scriptable shell you can use.

                     

                    You have done nothing to convince me that notarization is anything other than fall-of-a-log-easy. All you need to do is start with a valid executable. Apple seems to have been very lax about what constitutes a valid executable in the past. They seem to have tried hard to make sure that any old executable will run. Well, the modern notarization flow is more strict than that. Your app bundle needs to look like an app bundle. Your frameworks have to be real frameworks. Your info.plist has to be valid. Your dynamic libraries have to be properly linked. That all seems like a very reasonable starting point.

                • Re: Manual codesign / notarization and entitlements
                  eskimo Apple Staff Apple Staff (13,095 points)

                  While we investigate that problem, we wanted to add the appropriate entitlements to the signature, please tell me if there's something wrong with this

                  There’s something wrong with this (-:

                  The application-identifier entitlement is incorrect on the Mac; it should be com.apple.application-identifier [1].  However, that’s only part of that problem.  Those entitlements are restricted, meaning that they must be whitelisted by a provisioning profile.  My guess is that this is not what you want [2], in which case you should simply remove it.

                  It’s possible that there are other problems in play here, but I’m going to recommend that you fix this first and then let us know how that panned out.

                  Share and Enjoy

                  Quinn “The Eskimo!”
                  Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                  let myEmail = "eskimo" + "1" + "@apple.com"

                  [1] The exception being Mac Catalyst apps, which have both!

                  [2] Generally, you only need a profile if you want to use some other restricted entitlement, and com.apple.application-identifier comes along for the ride (it’s part of the mechanics that bind your code to your profile).

                    • Re: Manual codesign / notarization and entitlements
                      Saviq Level 1 Level 1 (0 points)

                      Hi eskimo,

                       

                      Yes, this was my (failed, ill-advised) attempt to avoid the hardened runtime from killing `hyperkit` because of:

                      CODE SIGNING: 31277[hyperkit] vm_map_protect can't have both write and exec at the same time  

                       

                      Is there some other way I can include an exemption for that binary? Or is notarization only currently possible with the hardened runtime enforced on all binaries?

                        • Re: Manual codesign / notarization and entitlements
                          eskimo Apple Staff Apple Staff (13,095 points)

                          Notarisation requires the hardened runtime.  The hardened runtime enables all sorts of extra security checks by default.  However, most (maybe even all?) of those can be disabled with hardened runtime exception entitlements.  These are not special entitlements (that is, entitlements that must be granted by Apple), nor are they restricted entitlements (that is, entitlements that must be whitelisted by a provisioning profile), but are completely open.  Any Developer ID code can use these entitlements without restrictions.

                          The problem with your original attempt is that you’re using a restricted entitlement (application-identifier) without a profile.  However, the other entitlement you used (com.apple.security.cs.disable-executable-page-protection) is not restricted, and you should be able to sign your binary with it.

                          As to whether that’ll fix your actual problem, it’s hard to say without knowing more about how your code is using vm_map_protect.

                          Share and Enjoy

                          Quinn “The Eskimo!”
                          Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                          let myEmail = "eskimo" + "1" + "@apple.com"