code signing blocked mmap() - how to find reason?

It seems like you have already fixed it. The only downside is that it makes your app more vulnerable to hacking and other types of exploitation. Think piracy rather than security. It is pretty clear what is causing it - Qt. Easily 70% of ALL notarization problems are caused by Qt.

Disable Library Validation Entitlement: "the app may load arbitrary plug-ins or frameworks, without requiring code signing"

... but all libs of my app bundle are code signed. I would like to know which unsigned lib triggers this problem.

You're right, deployment of Qt applications on macOS is a pain. If I'm using the prebuilt libs provided by the Qt company everything works fine. If I'm building the Qt libraries on my own (which is required in my project...) I get the mmap code sign error. So I'm quite sure that there must be solution without disabling library validation.

Has "disable-library-validation" any drawbacks?

Yes. Library validation is an important security feature (it prevents a whole world of exploits that involve loading malicious code into your process) and thus disabling it decreases security.

Will I still be able to notarize the app?

Yes.

How can I check what exactly is blocking the library?

This can be quite tricky. First up, when version of macOS are you testing this on?

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Review how your build scripts are different than the official ones for the Mac.

I found this link: doc.qt.io/archives/qtextended4.4/qtopia-resource-system.html

It seems to suggest that there is some SVG functionality that is associated with mmap() on Linux. Your build scripts may be enabling this code whereas the Qt builds omit it from the Mac platform. That's just a guess though. Building open source projects for the Mac can be really tricky. In some cases, it works better if there isn't an official Mac version. The "Mac code" is sometimes really hacked up with macros so it would build properly the last time someone worked on it for 10.5.

I'm building and testing the software with the latest macOS (10.15.4) running on the latest iMac, xCode is is up-to-date, too.

jogri emailed me a copy of their app, so I took a rummage inside.

To start, looking at the system log at the time of the failure revealed this:

type: error
time: 2020-05-05 21:53:29.092534 +0100
process: kernel
message: Library Validation failed: Rejecting '/Applications/MyApp.app/Contents/Frameworks/QtSvg.framework/Versions/5/QtSvg' (Team ID: none, platform: no) for process 'MyApp(581)' (Team ID: …, platform: no), reason: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

Clearly there’s something broken about the signature of the QtSvg framework. Let’s check that:

% codesign -v -vvv /Volumes/MyApp/MyApp.app/Contents/Frameworks/QtSvg.framework                 
/Volumes/MyApp/MyApp.app/Contents/Frameworks/QtSvg.framework: valid on disk
/Volumes/MyApp/MyApp.app/Contents/Frameworks/QtSvg.framework: satisfies its Designated Requirement
% codesign -d -vvv --entitlements :- /Volumes/MyApp/MyApp.app/Contents/Frameworks/QtSvg.framework
Executable=/Volumes/MyApp/MyApp.app/Contents/Frameworks/QtSvg.framework/Versions/Current/QtSvg
Identifier=org.qt-project.QtSvg
Format=bundle with Mach-O thin (x86_64)
CodeDirectory v=20200 size=2772 flags=0x0(none) hashes=81+3 location=embedded
Library validation warning=OS X SDK version before 10.9 does not support Library Validation
Hash type=sha256 size=32
CandidateCDHash sha256=e56113692b37fc138535c8f817e0f7479512cc7e
CandidateCDHashFull sha256=e56113692b37fc138535c8f817e0f7479512cc7ea496b807cd590f06fe70b566
Hash choices=sha256
CMSDigest=e56113692b37fc138535c8f817e0f7479512cc7ea496b807cd590f06fe70b566
CMSDigestType=2
CDHash=e56113692b37fc138535c8f817e0f7479512cc7e
Signature size=8929
Authority=Developer ID Application: …
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=5 May 2020 at 10:37:28
Info.plist entries=7
TeamIdentifier=…
Sealed Resources version=2 rules=13 files=3
Internal requirements count=1 size=180

So the library has a code signature but the system is refusing to load it. I’ve seen that before, and it’s actually hinted at by the warning on line 9 above, namely

Library validation warning=OS X SDK version before 10.9 does not support Library Validation

Consider this:

% otool -l /Volumes/MyApp/MyApp.app/Contents/Frameworks/QtSvg.framework/Versions/5/QtSvg | grep -B 1 -A 3 LC_VERSION_MIN_MACOSX
Load command 8
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.13
      sdk n/a

This library has a deployment target but no SDK version. That’s weird, and also problematic. The macOS code signing architecture changed in macOS 10.9, so certain code signing features are only available in 10.9 and later. The hardened runtime requires those features, and it checks for their presence by checking for your SDK version. This, btw, is the reason why notarisation requires a 10.9 or later SDK.

In short, your library has a bogus SDK value, which means that the hardened runtime is refusing to load it even though it’s signed correctly. Once you fix your SDK linkage, this problem should go away.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

You're right, the missing SDK version was the reason that triggered the error when launching the app. Quinn, thanks a lot for your help!

An error message like "Library rejected due to missing or outdated SDK version" instead of "code signing blocked mmap()" would have saved me a lot of time...

Two months ago I was able to create a lib with valid SDK version using these clang calls:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ \
-c \
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk \
-mmacosx-version-min=10.13 \
-o test.o \
test.cpp


/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ \
-Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk \
-mmacosx-version-min=10.13 \
-dynamiclib \
-o TestLib \
test.o

On my current system (I guess there was an Xcode update in the meantime?) the test.o still has "sdk 10.15.4" set, but the linker (second clang call) drops the SDK version. To fix this, I have to pass "-isysroot ..." to the linker call. Or I have to set the SDKROOT environment variable, or I have to launch the build process with "xcrun --sdk macos ...".

I've thought I can pass the SDK version as "syslibroot" to the linker, but maybe I'm wrong. I'm missing a good documentation how to build software on command line, without Xcode.