CMake, vcpkg, and universal builds

There are some hints about makefiles in the Apple silicon documentation.

Cmake, not make. And vcpkg. Alas.

I am very familiar with using Apple‘s tool, including Xcode and make, to build universal binaries of all sorts. I have been for a while. It’s this third-party stuff I need help with.

The important Clang compiler flags for cross-compiling are: --target; --sysroot; and -isysroot.

For CMake specifically, the relevant variables for cross-compiling are: CMAKE_C_COMPILER_TARGET; CMAKE_CXX_COMPILER_TARGET; CMAKE_SYSTEM_PROCESSOR; and CMAKE_SYSTEM_NAME.

For example, to build for x86_64 on an M1 Mac (arm64):

export TARGET="x86_64-apple-darwin"
export CFLAGS="$CFLAGS --target=$TARGET"
export CXXFLAGS="$CXXFLAGS --target=$TARGET"
export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"
cd mycode
mkdir build
cd build
cmake -DCMAKE_C_COMPILER_TARGET="$TARGET" -DCMAKE_CXX_COMPILER_TARGET="$TARGET" -DCMAKE_SYSTEM_PROCESSOR="x86_64" -DCMAKE_SYSTEM_NAME="Darwin" -DCMAKE_C_FLAGS="$CFLAGS" -DCMAKE_CXX_FLAGS="$CXXFLAGS" ..

Note if you set the SDKROOT environment variable, it is not necessary to add "--sysroot $SDKROOT" and "-isysroot $SDKROOT" to CFLAGS and CXXFLAGS.

Do that for each (x86_64 and arm64) then use lipo to make universal binaries.

Edit: Oh and, with CMake, never set the "--target" flag in LDFLAGS, only in CFLAGS/CXXFLAGS, it won't like that.

I'd never heard of vcpkg before you mentioned it.

I don't think you need to sign before doing lipo. You can wrap the whole thing into a script and easily do the lipo first and then sign the universal app.

I'm afraid the most likely explanation is that your build scripts need significant changes. I haven't encountered any cross platform incompatibilities on the several cmake-based projects I use. Getting cmake to generate iOS code is more difficult. There is a custom config script floating around that is designed specifically for iOS. (See https://fossies.org/linux/opencv/platforms/ios/cmake/Modules/Platform/iOS.cmake

Here are the custom configuration options I specify in my code:

CMAKE_INSTALL_PREFIX (You probably don't need this one)

CMAKE_C_COMPILER (Set to /path/to/Xcode/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang)

CMAKE_AR (Set to /path/to/Xcode/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar)

CMAKE_LINKER (Set to /path/to/Xcode/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld)

CMAKE_TOOLCHAIN_FILE (Set to the iOS.cmake file mentioned above, but only for iphoneos builds).

IOS_PLATFORM=OS (As opposed to SIMULATOR, again, only for iphoneos builds)

Otherwise, aside from being 8 hours too slow, I don't necessarily think you are doing anything wrong. I don't even use the above build settings anymore. One of my most important projects recently switched to cmake and that was the impetus I needed to set VERBOSE=1 and convert everything from automake, cmake, jam, ninja, and gn to Xcode. Now it works great and builds on all platforms, even in Xcode Cloud.

Over, on twitter, Alexander Neumann (@Iluinrandir) gave me the solution. I used

env PATH=${HOME}/vcpkg-arm64:${PATH} cmake -G Xcode .. -DVCPKG_HOST_TRIPLET=x64-osx -DVCPKG_TARGET_TRIPLET=arm64-osx -DCMAKE_TOOLCHAIN_FILE=${HOME}/vcpkg/scripts/buildsystems/vcpkg.cmake -DCMAKE_OSX_ARCHITECTURES=arm64 -DMACOS_BUILD_RELEASE=ON

to create the xcodeproj, and then time cmake --build . --config Release -- -allowProvisioningUpdates to build. And... I ended up with arm64 executables that work on my M1 MBP.

I'm not sure if the -DCMAKE_OSX_ARCHITECTURES=arm64 is necessary or not.

Now, this still leaves me wondering how to combine them into a single bundle. I mean, I know how to script the part where I lipo the various parts together, but I'm not sure how to get the proper signing done, other than copying the output of xcodebuild. We can have separate distributions for now (until and unless Apple lets me use the Endpoint Security Framework for real, we can't be feature-complete, or at least feature-parity with Windows, anyway).

I'm not sure what you mean about "copying the output of xcodebuild". What I've done in the past is save one output to "x86_64/myproj.framework" and the other architecture to "arm64/myproj.framework". Save yet another version to just "myproj.framework". Take any "executables" (tools or dylibs) from the two architecture-specific versions, lipo together, and write into the appropriate place in the universal version. Then sign the universal version.

However, I pay very close attention to all of my projects and check them carefully. Some open source projects will generate different build configurations on different platforms. If your projects are doing this, then you have no alternative. You must do two separate, architecture-specific builds.

In theory, a project could also save the architecture-specific information in public header files. Luckily, I haven't had to deal with that or I was able to hack around it. But this is something that has always been in the back of my mind when dealing with these kinds of build problems and hack-arounds.

I meant "copying the signing command." :)

I know this thread is kinda old, but since I've been googling around for a while, I thought I might as well share that I ended up using the method outlined in the blog post here: https://www.f-ax.de/dev/2022/11/09/how-to-use-vcpkg-with-universal-binaries-on-macos/

At least this explains how to get universal binaries out of vcpkg and there is a sample project for a universal cmake app... Maybe this is also helpful for other people searching for this issue...

Oh, that's very nice.

What we ended up doing was to build twice -- once for x86_64 and once for arm64. After both builds finish, I then have a script go through the set of executables I expect, and use lipo and codesign to create a universal bundle.

I might be missing something, or something has changed since last year, but for building universal binaries when resolving dependencies with vcpkg it is enough to pass both architectures to VCPKG_OSX_ARCHITECTURES in the triplet (so you might want to create a custom triplet for that):

set(VCPKG_TARGET_ARCHITECTURE arm64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)

set(VCPKG_CMAKE_SYSTEM_NAME Darwin)
set(VCPKG_OSX_ARCHITECTURES "arm64;x86_64")

With zstd as an example:

$ lipo -info ./libzstd.a
Architectures in the fat file: ./libzstd.a are: x86_64 arm64

I will have to try that! That would simplify things greatly!

Hm, no, I tried that, and it fails with openssl -- looks like it's trying to compile x86 assembly for arm64 architecture. 😩