Notarization: stapler error 65

This thread has been locked by a moderator.

Hello !


I'm trying to automatize notarization of a complex product composed of a bunch of components in preparation of macOS 10.14.5


The product is composed of a set of daemons, an app, and a kext. Everything use hardened runtime.


We build everything via a big Xcode workspace and xcodebuild command. To notarize, I create a zip archive of the resulting xcarchive, then I upload this zip archive to Apple server with altool --notarize-app.


It eventually finishes with success, and I receive the e-mail from Apple saying that my Mac software is ready to be distributed.


To be safe if users are offline, I want to staple the notarization ticket to my binaries.


I skip the daemons, because it seems it's not possible, for now, to attach a ticket to single Mach-O binaries (Error 73)


I staple the ticket to the kext : it works fine (and it's loaded even if the 10.14.5 machine is offline).


But when I try to staple the ticket to the app, I have this error :

Processing: /***/MyApplication.app
CloudKit query for MyApplication.app (2/[sha256]) failed due to "record not found".
Could not find base64 encoded ticket in response for 2/[sha256]
The staple and validate action failed! Error 65.


Should be noted that MyApplication.app and MyApplication.app/Contents/MacOS/MyApplication are in the final Apple logs (as all other Mach-O files in the archive). And [sha256] is a real valid sha256 value (just redacted there).


Should be noted too that if I zip directly MyApplication.app and upload this zip on Apple server with altool, then everything works as expected (I'm able to staple the notarization ticket to the application). So it seems to fail only when I notarize xcarchive, and only for .app bundles. I would still want to not do that, as each upload and notarization take time, so if it can be done in a single big step, it would be better.


And should be noted, finally, that MyApplication.app is inside a .bundle, while the kext is not (myarchive.xcarchive/Products/Library/TheKext.kext — myarchive.xcarchive/Products/Library/MyProduct/MyProduct.bundle/Contents/MacOS/MyApplication.app)


Any idea of what is happening there ?


Can it be related to https://forums.developer.apple.com/thread/115657 ?


Thank you !

Up vote post of evitcejbo
15k views
Posted by
evitcejbo
Reply
Add a Comment

Accepted Reply

Plus I'm doing exactly the same operations I've been doing successfully for several MONTHS.

It’s hard to be sure without looking at your case in more detail, but it’s likely that this was previously working because, since Sep 2019, the notary service has been treating errors as warnings. A few months later we announced that, in Feb 2020, it would resume treating errors as errors. If you notarised a product with an unsigned component a few months back and then tried to notarise exactly the same file again today, you’d see this problem.

I’m happy to help you investigate this. There’s two ways we can move forward here:

Finally, I’ve locked this thread and I want to explain why. The error that kicked off this thread, 

stapler
failing with error 65, has a very specific cause: 
stapler
has extracted the cdhash from the item you are trying to staple too, passed that to Apple’s server’s to look up its notarisation ticket, and has got a response saying that no such ticket exists. For example, to quote dev\@eptar.hu’s most recent post, the item they’re trying to staple has a cdhash of b4d5dbccccfdd15f5c4bd6c043a3684f7053f1fb but there’s no ticket that covers that cdhash.

As to why there’s no ticket for that cdhash, that’s a complex question. Assuming you actually submitted the item for notarisation, you can use the technique from my 23 Apr 2019 to determine exactly which cdhashes are included in your ticket. There’s two possibilities here:

  • The cdhash is actually in the ticket (A).

  • The cdhash isn’t in the ticket (B).

In case A, something wacky is happening. The only time I’ve seen this happen is where the developer was notarising the same product multiple times, and renaming the outermost app in between. This caused the notary service to replace the existing correct ticket with a new ticket that covered less than the original. If you see this problem — where the output from 

--notarization-info
shows that a ticket was issued by 
stapler
is unable to find it — you should definitely file a bug with the details. And if you want help with that, I recommend that you open a DTS tech support incident.

Case B is far more common. There have been a few cases where this was triggered by a bug in the notary service — for example, over on this thread it turned out that the notary service wasn’t handling universal binaries properly — but most of the time this is caused by a problem with the way the submitted product was formatted.

Regardless, my experience is that there’s no single cause of this error, so having everyone who encounters this error pile on to this thread just confuses matters, and that’s why I’ve locked it. If you see this problem, please start a new thread here in Distribution > Mac Apps and we can pick up things there.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware 

let myEmail = "eskimo" + "1" + "@apple.com"
Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Post marked as solved
Add a Comment

Replies

I just got the same error.


Never saw this error before.


My app is already notarized and inside its installer. Yet, after checking the log, the error claims that "The signature of the binary is invalid."


How the signature can be "invalid" if this binary is already NOTARIZED?


Plus I'm doing exactly the same operations I've been doing successfully for several MONTHS.


Now, my users are waiting for an important update which I can't post because of - yet another - bug in Apple's notarization process.



Apple's abuse of independent developers under pretence of "notarization" continues.


We, independent developers, who run our own businesses are not getting paid by the our. The waste of time due to this half-backed notarization thing has been unspeakable. Which for independent developers directly transpaltes into loss of income.



Leo

Posted by
Leo Braun
Up vote reply of Leo Braun Down vote reply of Leo Braun
Add a Comment

Plus I'm doing exactly the same operations I've been doing successfully for several MONTHS.

It’s hard to be sure without looking at your case in more detail, but it’s likely that this was previously working because, since Sep 2019, the notary service has been treating errors as warnings. A few months later we announced that, in Feb 2020, it would resume treating errors as errors. If you notarised a product with an unsigned component a few months back and then tried to notarise exactly the same file again today, you’d see this problem.

I’m happy to help you investigate this. There’s two ways we can move forward here:

Finally, I’ve locked this thread and I want to explain why. The error that kicked off this thread, 

stapler
failing with error 65, has a very specific cause: 
stapler
has extracted the cdhash from the item you are trying to staple too, passed that to Apple’s server’s to look up its notarisation ticket, and has got a response saying that no such ticket exists. For example, to quote dev\@eptar.hu’s most recent post, the item they’re trying to staple has a cdhash of b4d5dbccccfdd15f5c4bd6c043a3684f7053f1fb but there’s no ticket that covers that cdhash.

As to why there’s no ticket for that cdhash, that’s a complex question. Assuming you actually submitted the item for notarisation, you can use the technique from my 23 Apr 2019 to determine exactly which cdhashes are included in your ticket. There’s two possibilities here:

  • The cdhash is actually in the ticket (A).

  • The cdhash isn’t in the ticket (B).

In case A, something wacky is happening. The only time I’ve seen this happen is where the developer was notarising the same product multiple times, and renaming the outermost app in between. This caused the notary service to replace the existing correct ticket with a new ticket that covered less than the original. If you see this problem — where the output from 

--notarization-info
shows that a ticket was issued by 
stapler
is unable to find it — you should definitely file a bug with the details. And if you want help with that, I recommend that you open a DTS tech support incident.

Case B is far more common. There have been a few cases where this was triggered by a bug in the notary service — for example, over on this thread it turned out that the notary service wasn’t handling universal binaries properly — but most of the time this is caused by a problem with the way the submitted product was formatted.

Regardless, my experience is that there’s no single cause of this error, so having everyone who encounters this error pile on to this thread just confuses matters, and that’s why I’ve locked it. If you see this problem, please start a new thread here in Distribution > Mac Apps and we can pick up things there.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware 

let myEmail = "eskimo" + "1" + "@apple.com"
Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Post marked as solved
Add a Comment