Apple Notarization service failing on app that notarized successfully some weeks ago

What is the specific complaint? Please fetch the notary log and post an example of all the diagnostics related to a specific instance of this problem (that is, a specific Mach-O within a specific jar archive).

Feel free to redact stuff if you don’t want to share details. I outline my preferred redaction method in Posting a Crash Report; I like that because it still allows me to correlate identifiers in different places.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi,

I work with Buster, this is our notary log (please see below).

The complaint we are receiving is that the files are returning with "The binary is not signed." and "The signature does not include a secure timestamp.", etc. Despite our pipeline (and code) not changing between the successful and unsuccessful runs.

It only seems to fail on files that are contained within jars (again, despite working previously).

We have checked that our certificates are valid.

For future reference, is there anywhere we can see the notary service release notes? We had hypothesized a change on Apples side, requiring a change within our process, but could not find anything relevant.

Let us know how we can assist further towards fixing this.

Thanks, Bradley

We faced same issue. From the beginning of September our application bundle couldn't be notarised because of some .jnilib and .dylib was not signed. We tried notarise our previous release, which was notarised successfully before and got same error. When I verify signed application it returns it's valid

codesign --deep --verify --strict --verbose=3 Our_App.app
.....
Our_App.app: valid on disk
Our_App.app: satisfies its Designated Requirement

Can you say if it was changes in notarisation process on Apple notarisation service?

Hi Quinn,

We just started seeing this same issue, a new version of previously successful notarized was submitted and it fails to be notarized. The executable is an Install Anywhere installer that contains jar files and the jar files are the source of the notarization failure.

To test, I used the same unmodified earlier file, that successfully notarized, and submitted it for notarization again and it failed to notarize.

If acceptable, I can send the log files to your email as I'm prevented from letting the logs becoming public.

I have exactly the same issue. I've successfully notarized previous versions of my jpackage generated dmg (which contains my app jar as well as some third party jars and my app jar, in turn, contains some Mac, Linux, and Windows executables. Even though there have been no changes, the notarytool now marks my submission as invalid. When I look at the log, it looks like each Mac executable in my jar are marked with three errors:

1. The binary is not signed with a valid Developer ID certificate
2. The signature does not include a secure timestamp.
3. The executable does not have the hardened runtime enabled.

The errors are correct - I never did any of those things with these embedded executables. But until recently (is it perhaps the move to macOS 18??) the notary tool never rejected my dmg because of these.

Sorry about not replying sooner; DevForums has an ongoing problem where it fails to notify me of thread updates (r. 131905835)-:

Hmmm, lemme start with an explanation of what you should do, and then come back to the history.

All native code [1] included in a Developer ID product must be signed and notarised. If the code isn’t included in the ticket issued by the notary service, the system won’t run it. Notarisation Fundamentals explains the how and why of this in more detail.

If the native code is embedded in an archive:

• You must use an archive format that the notary service understands.

• The code must be signed appropriately.

I discuss this in detail in the Executable Code Inside Archives section of Notarisation Fundamentals.

If the notary service is complaining about the native code within an archive, you have two options:

• Unpack the archive, sign the code, and repack it.

• Remove the code from the archive.

Note Don’t discount that last point. If this native code wasn’t previously being covered by your notarised ticket, meaning it could never be executed, and your product actually worked, it’s quite possible that you don’t need that code at all.

Coming back to the history of this, it’s clear that something changed with the notary service recently. The weird thing is that I remember the notary service previously rejecting JAR archives containing code that wasn’t correctly signed. That’s why I included a discussion of this in Notarisation Fundamentals, which dates from 2022. Indeed, I looked through my records and found this post, where I explained this process all the way back in 2020.

So it’s possible we’re in a “gone back to metric” situation [2], where the notary service was previously enforcing this, then stopped, and is now enforcing it again. Regardless of that, the path forward here is the same: Sign and notarise your native code.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] In this context I’m using the term native code to refer to a Mach-O image. This is a binary containing code that the system can load and run directly. See An Apple Library Primer for an explicit definition of that term.

[2] https://www.youtube.com/watch?v=JB6k-fatJmI

Hi Quinn,

No problem, its good to hear from you!

For some additional context we are unpacking jars and signing them before repacking. We rely on these libraries for various functions within the program and can't just remove the code.

For example we are using jffi-1.3.10-native.jar for which we need to sign the libjffi-1.2.jnilib binary. Our code was doing this correctly but as of September 4th we are now unable to successfully sign these.

We have found various PR's from a few years ago about Apple dropping support for java compiled against MacOS 10.9 and earlier. Is there possibly something to do with this and is there any official docs we can read in order to keep up to date with these changes if so?

Thanks, Bradley

We have found various PR's from a few years ago about Apple dropping support for java compiled against MacOS 10.9 and earlier.

That’s probably the issue that I talk about in Notarisation and the macOS 10.9 SDK. That’s not a new change — it’s been in place since the Dawn of Time™ — but it’s possible that you’re now hitting it.

Is there possibly something to do with this … ?

It’s certainly possible. The way to tell is to run vtool over the Mach-O image itself. For example:

% vtool -show-build /Applications/NetNewsWire.app/Contents/Frameworks/Sparkle.framework/Versions/A/Sparkle 
/Applications/NetNewsWire.app/Contents/Frameworks/Sparkle.framework/Versions/A/Sparkle (architecture x86_64):
Load command 8
      cmd LC_VERSION_MIN_MACOSX
…
      sdk 11.1
/Applications/NetNewsWire.app/Contents/Frameworks/Sparkle.framework/Versions/A/Sparkle (architecture arm64):
Load command 9
      cmd LC_BUILD_VERSION
…
 platform MACOS
…
      sdk 11.1
…

This confirms that both ‘slices’ of the Sparkle framework were built with the macOS 11.1 SDK.

If you have a specific JAR that is failing notarisation even though you’ve signed it correctly, I’m happy to take a look. Just post:

• A link to the JAR — You may need to do this in the clear, per tip 14 in Quinn’s Top Ten DevForums Tips.

• A copy of the notary log you got back — Include that as a text attachment, like you did in your earlier post.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi Quinn,

The jar we are trying to sign is jffi from https://github.com/jnr/jffi/releases/tag/jffi-1.3.10. We are extracting the libjffi-1.2.jnilib from the jar and signing before re-packing them. This step goes fine with nothing going wrong. We then get issues when trying to notarize the application where it says the binary isn't signed, the signature isn't valid and there isn't a valid timestamp.

sign-jar-force "./Contents/Resources/xx/drivers/jffi-1.3.10-native.jar" "jni/Darwin/libjffi-1.2.jnilib"

sign-jar()
{
    if [ -e $"$app_path/$1" ]; then
        echo "Signing (jar): $1 $2"
        unzip "$app_path/$1" "$2" -d "$temp_path" || echo "(jar $1): unzipping $2 failed" > "$temp_path/.failed"
        codesign --sign "$sign_identity" --timestamp "$temp_path/$2" || echo "(jar $app_path/$1): $temp_path/$2" > "$temp_path/.failed"
        jar -ufv "$app_path/$1" "$temp_path/$2" || echo "(jar $1) zipping $temp_path/$2" > "$temp_path/.failed"
        rm -f "$temp_path/$2"
    else
        echo "Skipping: $1 (path not found)"
    fi
}

This is our method for signing the jar. We have also tried it with putting --force on the codesign method.

Thanks, Bradley

The jar we are trying to sign is jffi from

Is there a place to download the actual JAR from there? Because both of the downloads on that page are labelled as “Source code”. And I’m not set up to build and package Java code.

Ideally I’d like to get:

• A link to the JAR that failed notarisation

• A notary log that for that failure

This doesn’t have to be your entire app. You could, for example, create a zip archive of just that JAR and try notarising that [1].

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] And if that doesn’t reproduce the problem then that’s an interesting data point in and of itself.

Good Morning,

Sorry about that, we are using Maven so I believe this would be the link where the Jar we are getting is from https://repo1.maven.org/maven2/com/github/jnr/jffi/1.3.10/ , specifically the "jffi-1.3.10-native.jar".

Thanks, Bradley

we are using Maven so I believe this would be the link where the Jar we are getting is from …

Ta!

I downloaded that, unzipped it, and looked at the signature:

% codesign -d -vvv --entitlements - jffi-1.3.10-native/jni/Darwin/libjffi-1.2.jnilib
…
CodeDirectory v=20400 size=1547 flags=0x2(adhoc) hashes=42+2 location=embedded
Hash type=sha256 size=32
…

That looks reasonable enough. It’s ad hoc signed, which is pretty typical for this sort of thing. It has modern hashes, which is good. And it has no entitlements, which is also good.

It also has a reasonable SDK value:

% vtool -show-build jffi-1.3.10-native/jni/Darwin/libjffi-1.2.jnilib
jffi-1.3.10-native/jni/Darwin/libjffi-1.2.jnilib (architecture x86_64):
Load command 8
…
  version 10.6
      sdk 12.0
jffi-1.3.10-native/jni/Darwin/libjffi-1.2.jnilib (architecture arm64):
Load command 9
…
    minos 11.0
      sdk 12.0
…

Note The Intel deployment target is a bit… well… optimistic, but I don’t think that’s going to cause problems.

So I re-signed the code:

% codesign -s "Developer ID Application" -f --timestamp "jffi-1.3.10-native/jni/Darwin/libjffi-1.2.jnilib"
…/libjffi-1.2.jnilib: replacing existing signature

And packaged it up into a zip archive:

% ditto -c -k jffi-1.3.10-native jffi-1.3.10-native.zip

Note I don’t have JAR tools installed )-: so I’m just using ditto.

And then notarised that:

% xcrun notarytool submit …credentials… --wait jffi-1.3.10-native.zip 
…
  id: f3601c63-1b99-40e7-a628-5ab4170f0430
  status: Accepted

And that worked. Nice. I then fetched the notary log:

% notarytool-log f3601c63-1b99-40e7-a628-5ab4170f0430
{
  "logFormatVersion": 1,
  "jobId": "f3601c63-1b99-40e7-a628-5ab4170f0430",
  "status": "Accepted",
  "statusSummary": "Ready for distribution",
  "statusCode": 0,
  "archiveFilename": "jffi-1.3.10-native.zip",
  "uploadDate": "2024-10-15T11:45:30.111Z",
  "sha256": "260ab999bd128fe71a9085fc728f8f5b7e403aedc48f12cd2de9697a15531994",
  "ticketContents": [
    {
      "path": "jffi-1.3.10-native.zip/jni/Darwin/libjffi-1.2.jnilib",
      "digestAlgorithm": "SHA-256",
      "cdhash": "e96f74198d758dc2b2c0512284042b70e47961e3",
      "arch": "x86_64"
    },
    {
      "path": "jffi-1.3.10-native.zip/jni/Darwin/libjffi-1.2.jnilib",
      "digestAlgorithm": "SHA-256",
      "cdhash": "d82f596b847d3cfdde701b32d3604e9facf4f008",
      "arch": "arm64"
    }
  ],
  "issues": null
}

There are no errors and the log contains cdhash entries for both architectures in the Mach-O that I submitted:

% codesign -d -vvv --arch x86_64 jffi-1.3.10-native/jni/Darwin/libjffi-1.2.jnilib
…
CDHash=e96f74198d758dc2b2c0512284042b70e47961e3
…
% codesign -d -vvv --arch arm64 jffi-1.3.10-native/jni/Darwin/libjffi-1.2.jnilib 
…
CDHash=d82f596b847d3cfdde701b32d3604e9facf4f008
…

In summary, all of this behaved as I expected. I’m not sure why you’re hitting problems, but that .jnilib is definitely acceptable to the notary service once you sign it for distribution.

I’m gonna let you digest the above and then run some more tests at your end. If you continue to have problems, post back here and we can talk about next steps.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi,

We found the solution in our pipeline which turned out to be using ditto in place of jar. We think when repackaging using jar this was modifying the file and therefore removing its signature.

Unsure as to why this has hit us now, we couldn't find any material online that hints to Apples requirements getting stricter to include these unsigned files.

Thanks for all the help, Bradley