Use dyld to link in frameworks at runtime. Use ld to make your programs and link archive libraries at build time.

Posts under Linker tag

129 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Need Assistance with xCode Project - Linker Errors and Version Compatibility with Apple External Purchase Link
Hello Apple Developer Community, I am currently working on an xCode project that uses Objective-C and I am trying to integrate Apple's external purchase link APIs available in iOS 15.4. To use these APIs, I need to utilize async and completion handlers which are available from iOS 13 onwards. The issue I'm facing is that the current minimum deployment version for my project is iOS 12. I would like to ensure my code compiles properly for the correct iOS versions and I am also using #if available; to ensure this. However, I am unsure of how to guarantee my code builds effectively with the deployment version being set to either iOS 12 or 13. I am currently facing the following linker errors: ld: warning: Could not find or use auto-linked library 'swiftCompatibility56': library 'swiftCompatibility56' not found ld: warning: Could not find or use auto-linked library 'swiftCompatibilityConcurrency': library 'swiftCompatibilityConcurrency' not found ld: warning: Could not find or use auto-linked library 'swiftCompatibilityPacks': library 'swiftCompatibilityPacks' not found ld: Undefined symbols: __swift_FORCE_LOAD_$_swiftCompatibility51, referenced from: __swift_FORCE_LOAD_$_swiftCompatibility51_$_ProjectXYZ in ProjectXYZ.a[20](ExternalPurchaseLinkWrapper.o) __swift_FORCE_LOAD_$_swiftCompatibility56, referenced from: __swift_FORCE_LOAD_$_swiftCompatibility56_$_ProjectXYZ in ProjectXYZ.a[20](ExternalPurchaseLinkWrapper.o) __swift_FORCE_LOAD_$_swiftCompatibilityConcurrency, referenced from: __swift_FORCE_LOAD_$_swiftCompatibilityConcurrency_$_ProjectXYZ in ProjectXYZ.a[20](ExternalPurchaseLinkWrapper.o) clang: error: linker command failed with exit code 1 (use -v to see invocation) I would appreciate any guidance or advice on this matter. How do I ensure my code compiles and works only for the correct versions and how do I resolve the linker errors? Thank you in advance for your assistance. Best, Sumanth
1
0
486
May ’24
Determining Why a Symbol is Referenced
Recently a bunch of folks have asked about why a specific symbol is being referenced by their app. This is my attempt to address that question. If you have questions or comments, please start a new thread. Tag it with Linker so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Determining Why a Symbol is Referenced In some situations you might want to know why a symbol is referenced by your app. For example: You might be working with a security auditing tool that flags uses of malloc. You might be creating a privacy manifest and want to track down where your app is calling stat. This post is my attempt at explaining a general process for tracking down the origin of these symbol references. This process works from ‘below’. That is, it works ‘up’ from you app’s binary rather than ‘down’ from your app’s source code. That’s important because: It might be hard to track down all of your source code, especially if you’re using one or more package management systems. If your app has a binary dependency on a static library, dynamic library, or framework, you might not have access to that library’s source code. IMPORTANT This post assumes the terminology from An Apple Library Primer. Read that before continuing here. The general outline of this process is: Find all Mach-O images. Find the Mach-O image that references the symbol. Find the object files (.o) used to make that Mach-O. Find the object file that references the symbol. Find the code within that object file. This post assumes that you’re using Xcode. If you’re using third-party tools that are based on Apple tools, and specifically Apple’s linker, you should be able to adapt this process to your tooling. If you’re using a third-party tool that has its own linker, you’ll need to ask for help via your tool’s support channel. Find all Mach-O images On Apple platforms an app consists of a number of Mach-O images. Every app has a main executable. The app may also embed dynamic libraries or frameworks. The app may also embed app extensions or system extensions, each of which have their own executable. And a Mac app might have embedded bundles, helper tools, XPC services, agents, daemons, and so on. To find all the Mach-O images in your app, combine the find and file tools. For example: % find "Apple Configurator.app" -print0 | xargs -0 file | grep Mach-O Apple Configurator.app/Contents/MacOS/Apple Configurator: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64] … Apple Configurator.app/Contents/MacOS/cfgutil: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64] … Apple Configurator.app/Contents/Extensions/ConfiguratorIntents.appex/Contents/MacOS/ConfiguratorIntents: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64] … Apple Configurator.app/Contents/Frameworks/ConfigurationUtilityKit.framework/Versions/A/ConfigurationUtilityKit: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64] … This shows that Apple Configurator has a main executable (Apple Configurator), a helper tool (cfgutil), an app extension (ConfiguratorIntents), a framework (ConfigurationUtilityKit), and many more. This output is quite unwieldy. For nicer output, create and use a shell script like this: % cat FindMachO.sh #! /bin/sh # Passing `-0` to `find` causes it to emit a NUL delimited after the # file name and the `:`. Sadly, macOS `cut` doesn’t support a nul # delimiter so we use `tr` to convert that to a DLE (0x01) and `cut` on # that. # # Weirdly, `find` only inserts the NUL on the primary line, not the # per-architecture Mach-O lines. We use that to our advantage, filtering # out the per-architecture noise by only passing through lines # containing a DLE. find "$@" -type f -print0 \ | xargs -0 file -0 \ | grep -a Mach-O \ | tr '\0' '\1' \ | grep -a $(printf '\1') \ | cut -d $(printf '\1') -f 1 Find the Mach-O image that references the symbol Once you have a list of Mach-O images, use nm to find the one that references the symbol. The rest of this post investigate a test app, WaffleVarnishORama, that’s written in Swift but uses waffle management functionality from the libWaffleCore.a static library. The goal is to find the code that calls calloc. This app has a single Mach-O image: % FindMachO.sh "WaffleVarnishORama.app" WaffleVarnishORama.app/WaffleVarnishORama Use nm to confirm that it references calloc: % nm "WaffleVarnishORama.app/WaffleVarnishORama" | grep "calloc" U _calloc The _calloc symbol has a leading underscore because it’s a C symbol. This convention dates from the dawn of Unix, where the underscore distinguish C symbols from assembly language symbols. The U prefix indicates that the symbol is undefined, that is, the Mach-O images is importing the symbol. If the symbol name is prefixed by a hex number and some other character, like T or t, that means that the library includes an implementation of calloc. That’s weird, but certainly possible. OTOH, if you see this then you know this Mach-O image isn’t importing calloc. IMPORTANT If this Mach-O isn’t something that you build — that is, you get this Mach-O image as a binary from another developer — you won’t be able to follow the rest of this process. Instead, ask for help via that library’s support channel. Find the object files used to make that Mach-O image The next step is to track down which .o file includes the reference to calloc. Do this by generating a link map. A link map is an old school linker feature that records the location, size, and origin of every symbol added to the linker’s output. To generate a link map, enable the Write Link Map File build setting. By default this puts the link map into a text (.txt) file within the derived data directory. To find the exact path, look at the Link step in the build log. If you want to customise this, use the Path to Link Map File build setting. A link map has three parts: A simple header A list of object files used to build the Mach-O image A list of sections and their symbols In our case the link map looks like this: # Path: …/WaffleVarnishORama.app/WaffleVarnishORama # Arch: arm64 # Object files: [ 0] linker synthesized [ 1] objc-file [ 2] …/AppDelegate.o [ 3] …/MainViewController.o [ 4] …/libWaffleCore.a[2](WaffleCore.o) [ 5] …/Foundation.framework/Foundation.tbd … # Sections: # Address Size Segment Section 0x100008000 0x00001AB8 __TEXT __text … The list of object files contains: An object file for each of our app’s source files — That’s AppDelegate.o and MainViewController.o in this example. A list of static libraries — Here that’s just libWaffleCore.a. A list of dynamic libraries — These might be stub libraries (.tbd), dynamic libraries (.dylib), or frameworks (.framework). Focus on the object files and static libraries. The list of dynamic libraries is irrelevant because each of those is its own Mach-O image. Find the object file that references the symbol Once you have list of object files and static libraries, use nm to each one for the calloc symbol: % nm "…/AppDelegate.o" | grep calloc % nm "…/MainViewController.o" | grep calloc % nm "…/libWaffleCore.a" | grep calloc U _calloc This indicates that only libWaffleCore.a references the calloc symbol, so let’s focus on that. Note As in the Mach-O case, the U prefix indicates that the symbol is undefined, that is, the object file is importing the symbol. Find the code within that object file To find the code within the object file that references the symbol, use the objdump tool. That tool takes an object file as input, but in this example we have a static library. That’s an archive containing one or more object files. So, the first step is to unpack that archive: % mkdir "libWaffleCore-objects" % cd "libWaffleCore-objects" % ar -x "…/libWaffleCore.a" % ls -lh total 24 -rw-r--r-- 1 quinn staff 4.1K 8 May 11:24 WaffleCore.o -rw-r--r-- 1 quinn staff 56B 8 May 11:24 __.SYMDEF SORTED There’s only a single object file in that library, which makes things easy. If there were a multiple, run the following process over each one independently. To find the code that references a symbol, run objdump with the -S and -r options: % xcrun objdump -S -r "WaffleCore.o" … ; extern WaffleRef newWaffle(void) { 0: d10083ff sub sp, sp, #32 4: a9017bfd stp x29, x30, [sp, #16] 8: 910043fd add x29, sp, #16 c: d2800020 mov x0, #1 10: d2800081 mov x1, #4 ; Waffle * result = calloc(1, sizeof(Waffle)); 14: 94000000 bl 0x14 <ltmp0+0x14> 0000000000000014: ARM64_RELOC_BRANCH26 _calloc … Note the ARM64_RELOC_BRANCH26 line. This tells you that the instruction before that — the bl at offset 0x14 — references the _calloc symbol. IMPORTANT The ARM64_RELOC_BRANCH26 relocation is specific to the bl instruction in 64-bit Arm code. You’ll see other relocations for other instructions. And the Intel architecture has a whole different set of relocations. So, when searching this output don’t look for ARM64_RELOC_BRANCH26 specifically, but rather any relocation that references _calloc. In this case we’ve built the object file from source code, so WaffleCore.o contains debug symbols. That allows objdump include information about the source code context. From that, we can easily see that calloc is referenced by our newWaffle function. To see what happens when you don’t have debug symbols, create an new object file with them stripped out: % cp "WaffleCore.o" "WaffleCore-stripped.o" % strip -x -S "WaffleCore-stripped.o" Then repeat the objdump command: % xcrun objdump -S -r "WaffleCore-stripped.o" … 0000000000000000 <_newWaffle>: 0: d10083ff sub sp, sp, #32 4: a9017bfd stp x29, x30, [sp, #16] 8: 910043fd add x29, sp, #16 c: d2800020 mov x0, #1 10: d2800081 mov x1, #4 14: 94000000 bl 0x14 <_newWaffle+0x14> 0000000000000014: ARM64_RELOC_BRANCH26 _calloc … While this isn’t as nice as the previous output, you can still see that newWaffle is calling calloc.
0
0
361
May ’24
Mergeable Libraries issue with workspace subproject products.
Xcode version: 15.2, iOS deployment target 15.0 When experimenting with the new Xcode 15/new linker feature "Mergable Libraries" I came across an issue with subproject framework products. I have a workspace with two projects. I want to create a group framework named GroupFramework that consists of three other frameworks. FrameworkA, FrameworkB and FrameworkC. FrameworkA and FrameworkB are in the same project as the group framework I want to create. FrameworkC is a product of another project in my workspace. GroupFramework has all three frameworks linked in the "link binary with libraries" build phase. GroupFramework has "Create merged binary" set to manual because I want to control which frameworks are linked. FrameworkA, FrameworkB and FrameworkC all have "Build mergeabe library" set to YES. The application builds in both RELEASE and DEBUG. Running in DEBUG and RELEASE When running the application on a device in DEBUG or RELEASE it crashes with a linker error because FrameworkC can't be found on disk. When inspecting the app binary in RELEASE I can see that FrameworkA and FrameworkB are both in the ReexportedBinary folder, but FrameworkC is nowhere to be seen. I have tried embedding FrameworkC explicitly in the settings of the GroupFramework essentially creating an umbrella framework, and that also does not fix the issue. I have also tried embedding FrameworkC directly into the AppTarget that will consume GroupFramework. This solves the linking issue but a new issue regarding unresolved symbols then crops up. It appears as though the Reexporting and Merging of subproject frameworks is not working compared to a product that exists in the same project or at least may require a lot more manual configuration which is not currently documented anywhere. If this is the case, it could be very problematic when trying to adopt mergeable libraries in multi-project workspaces.
1
0
785
May ’24
Accessing the macOS Dynamic Linker (dyld)
The documentation about the Disable Library Validation Entitlement mentioned that the macOS dynamic linker (dyld) provides a detailed error message when the system prevents code from loading due to library validation. You can find more information here: https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_disable-library-validation I need assistance in locating the dynamic linker (dyld) on macOS Ventura 13.0. What are the various methods available to locate it? How can I access or open it for reading? Additionally, do I need any external tools to facilitate this process? My ultimate goal is to examine the detailed error message to identify any issues I am encountering with my application. Additionally, I have found one at /usr/lib/dyld, but it's not human-readable, nor does it have timestamps for whatever is logged. Based on my findings, I should be able to locate dyld at System/Library, but I can't find it there either.
2
0
421
Apr ’24
creating python modules that can call back to python
I'm doing the python dynamic module loading dance. As you likely know, this is not the typical shared library scenario described on all the docs, where an executable is linked against an existing shared lib. Instead cythonized python modules need to call back to the interpreter that loaded them for various functions. Thus, when creating the interpreter, some manner of symbol export must happen, which can then later be linked into a module's dynamic lib to resolve interpreter symbols. With GNU/GCC this is all magic with the -shared flag (perhaps GNU's weak symbol feature, unsupported on macOS, helps out?). With a MingW GCC variant on Windows -Wl,--output-def python.def is required, followed by dlltool -d python.def -l python.dll.a, which creates some object thingy that can be linked into the subsequent module .so files. I played with lipo python -create -output libpython.dylib on macOS, but saw that no one else uses this and abandoned it. The python3 on Darwin uses clang -bundle -undefined dynamic_lookup, but when I roll my own custom interpreter, building a module by hand with the above gives: ld: warning: -undefined dynamic_lookup may not work with chained fixups ... whatever that means. Link args of -bundle -bundle_loader ../python result in NO errors or warnings, but my interpreter cannot load these module .so files either, probably due to some error I have not yet unmasked. I am hoping to find a solution that works for both Xcode and Homebrew, and that will last for a decade or so, but for now would be happy getting something to work today.
2
0
356
Apr ’24
Mergeable Libraries: Missing Resources
It's not possible to merge a framework with resource into an iOS app target because the resource are not included in the app bundle. Steps to reproduce: Create an Xcode Project with iOS App Template Add a Framework Target (make sure to "Embed in Application") Add an Asset Catalog to Framework Target Add an Color Resource (or Image Set, or any other Resource) Reference the Resource in App Target (I have used a SwiftUI View) Run on Device (!) to make sure everything works as expected Change "Create Merged Binary (MERGED_BINARY_TYPE)" build setting of app target to "Automatic (automatic)" Change app target settings to link, but not embed framework target (e.g. change from "Embed and Sign" to "Do Not Embed" in "Frameworks, Libraries and Embedded Content" section in "General" tab) Run again (on Device!) and observe how the resources framework resource cannot be found anymore (using SwiftUI you will see a "No image/color named '...' in asset catalog for ..." error message in console logs) Note: Everything works fine in Simulator Same behavior for Release and Debug configuration Same behavior for manual and automatic merging Same behavior for resources which are not bundled in Asset Catalog When archiving the app, an "Assets.car" file is never present (even when creating archiving for Simulator target, when "Allow archiving for Simulator" is enabled) Reported as FB13716505 Test Project: https://github.com/iteracticman/MergeableResources/
7
1
1.3k
Oct ’24
Xcode 15.2 linker error
Xcode 15.2 linker error (Assertion failed: (rebasePtr->target == low56), function writeChainEntry, file ChainedFixups.cpp, line 1218) The same project builds fine on Xcode 14.3.1. It also builds fine when building a debug build to device. Any ideas on what we might be doing wrong?
1
0
513
Apr ’24
Apply Fixups taking too long during iOS app launch
Hello everyone, Our iOS app is taking too long to launch. On checking the launch profile, we are seeing that most of the launch time is being spent in applying fixups which is taking more than a second and at times even more to complete. Our deployment target is iOS 15+. We have checked using dyld_info that our binary uses chained fixups. Since chained fixups are enabled, page-in linking should also be enabled for our app as per this WWDC session. Can someone please help us understand why the fixups application is taking this long and how can we improve it? Thanks.
1
0
788
Apr ’24
Page-in Linking flag
Hi everyone, Our app runs on iOS deployment target 15.0. As per the WWDC 2022 session, page-in linking should be enabled for apps where chained-fixups is enabled. Since chained fixups are enabled from iOS deployment target 13.4+ which is true for our app, will page-in linking be applied to it? Is there some flag which we can use to confirm if it is applied for our app or not? We have tried using dyld_info tool but could not find a command which can provide this information.
1
0
551
Apr ’24
dyld Symbol Not Found on Some Devices
I have a React-Native App that I am trying to build to iOS using Xcode. When I build to my iPhone 12 Mini (iOS 17.4.1), the app works perfectly. When I build to my iPhone 7 Plus (iOS 15.8.2), the app pauses running immediately, and Xcode displays the following in the log: dyld[935]: Symbol not found: (_JSGlobalContextSetInspectable) Referenced from: '/private/var/containers/Bundle/Application/2579192B-74C5-4B54-AA59-948C49A4A7CA/MANHUNT2.app/MANHUNT2' Expected in: '/System/Library/Frameworks/JavaScriptCore.framework/JavaScriptCore' The app used to work perfectly on both phones, but this error has been happening recently, and I am unsure of the cause. Cleaning build folder and deleting derived data has not fixed the issue. Xcode: Version 15.3 React-Native: 0.73.6 Working Phone: iOS 17.4.1 Non-Working Phone: iOS 15.8.2
4
2
2.8k
Apr ’24
Guidance on splitting iOS app into static frameworks with assets
Hi, I am splitting my iOS app into smaller components, the natural way of doing this is to create static libraries that I can link with the main binary. I did succeed at making small static libraries (.a) and link them, but I was struggling with static framework where I have color assets. Finally I was able to make it work by going to the app target > Build Phases > Copy Bundle Resources and add there the .xcassets from the static framework I created. The confusion on my side is that the assets are accessed from the Framework code not from the app target code, and the guidance I am asking for is: How to create and link a static framework on iOS so that the framework can load and use it's own defined assets without having to do the Copy Bundle Resources step. Best Regards, Ion
1
0
491
Apr ’24
Undefined symbol: __mh_execute_header
I am building a unit test. The target of this unit test is my framework. My framework linked with couple of static libraries. When the test is running in simulator with host app it works fine. But when I try to test without host app I get following error from linker. PS: My test's xcconfig is similar to the host app's, and I can build the host app successfully. Undefined symbols for architecture arm64: "__mh_execute_header", referenced from: So what is __mh_execute_header and what framework/library it is defined in? Thanks for any help!
1
0
1.3k
Apr ’24
XCode 15 linker ld_prime causing new dependencies that don't work
With XCode 14 and below, our macOS application is building and running fine. The application depends on some third-party .dylibs and also a couple of .frameworks that we make in-house, all of which are happily in the .app bundle and working. When compiled under XCode 15, the application crashes because one of our .frameworks now tries to load these dylibs which cannot be found if run on a machine that didn't build it. Note that previously, this framework never depended on these dylibs before, but the application does. The rpaths for these dylibs are pointing to the build dependencies folder, which would only exist on a build machine. Also, the dependencies are now looking for versioned dylib filenames, while the application itself depends on the un-versioned dylib filenames. So to recap, that's 3 new problems when building with the new linker: The framework is now dependent on dylibs that normally only the application depends on The new dependency is on versioned dylibs, where the original application dependency is on un-versioned dylibs The framework's rpaths now include intermediate build folders Running otool -L on the framework binary shows a clear difference between ld_prime and ld_classic. The dependencies on the third-party dylibs are only showing up when the framework is built with the new linker. We have a couple of workarounds: Using ld_classic to build our application with the old linker Using post-build commands (install_name_tool) to change the dependencies and remove the build folders from the rpaths Though ultimately, workaround 1) could only be temporary and 2) is hacky. We'd like to understand why the linker is exhibiting this new behaviour, so we can make the proper adjustments to fix this the correct way. Any insight would be greatly appreciated. Thanks in advance. Note: this is a cross-platform product, and CMake is used to generate the .xcodeproj project file.
1
0
706
Apr ’24
Mergeable Libraries vs. Static Libraries
After reading An Apple Library Primer and watching Meet mergeable libraries (both really good resources) I'm sold on mergeable libraries. In my company we're working with static libraries as much as possible (both for precompiled dependencies and for our own Xcode projects). I can see that switching to mergeable libraries would (at least) improve iteration time because mergeable libraries would only be merged in release configs by default. You made it sound like it's "the holy grail of libraries". However, before starting the migration process I'd like to know: Is there any drawback to (merged) mergeable libraries when compared to using static libraries? I got the idea that mergeable libraries behave "almost" like static libraries after being linked "statically" with the app. What differences can I expect? Is there any situation where it's better to use static libraries instead of mergeable libraries? Thank you!
1
0
584
Mar ’24
building for iOS, but linking in object file built for macOS, file 'btfs.a' for architecture arm64
Hello there, I'm struggling with the following issue since Xcode 15 arrived and haven't been able to solve it. I have a go library c-archive library which I'm using in a iOS app and build were OK until Sonoma updated came last year. I updated all my go libraries and go runtime to latest version and still facing issues: my make script as follows: ios-arm64: CGO_ENABLED=1 GOOS=ios GOARCH=arm64 SDK=iphoneos SDK_PATH=xcrun --sdk iphoneos --show-sdk-path CARCH="arm64" CC=$(PWD)/clangwrap.sh CGO_CFLAGS="-fembed-bitcode" go build -v -buildmode=c-archive -ldflags="-s -w" -gcflags=all="-l -B" -tags ios -o $(IOS_OUT)/btfs.a . If there is anything else you need I can share more details. Thanks!
1
0
767
Mar ’24