Use dyld to link in frameworks at runtime. Use ld to make your programs and link archive libraries at build time.

Posts under Linker tag

129 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Is it possible to build a dynamic framework without the symbols of a static library that it links to?
I have an app that links to two dynamic frameworks which both link to the same static library, as follows: |--App |--DynamicFramework1 |--StaticLibrary |--DynamicFramework2 |--StaticLibrary <- the same library that DynamicFramework1 links to The static library's symbols are included in each framework's binary because of the way dynamic frameworks are built by default. The app therefore finds duplicates of the static library's symbols at runtime. Is it possible to link a dynamic framework to a static library (and to still be able to call on the classes and methods of the static library within the dynamic framework) in a manner that symbols from the static library are excluded from the dynamic framework's binary? My hope in doing this is that the binary of each of the two dynamic frameworks will exclude the symbols of the static library. I will then make it the responsibility of the app to link to the static library directly. Notes I have tried linking my dynamic framework with the static library in two different ways thus far: (1) I added the static library to my framework's "Link Binary with Libraries" Build Phase; and (2) I referenced the static library in my framework's "Other Linker Flags" Build Setting. Both result in the static library's symbols being included in the framework's binary. I am aware that changing a framework target's "Mach-O Type" from "Dynamic Framework" to "Static Library" will build the framework's binary without the symbols of the static libraries that it links to. I want to keep my frameworks as dynamic frameworks so that (1) I can benefit from how Xcode bundles together resources (strings, storyboards etc) automatically for dynamic frameworks; and (2) users of my framework can benefit from Mergeable Libraries in the near future. I am aware that I can solve this problem by changing the static library to a dynamic framework. I want to avoid this as much as possible since the static library is from a third-party. I want to avoid forking the static library's source code and messing with its build scripts if I can.
4
1
1.9k
Mar ’24
dyld not searching in /usr/local/lib (cmd line tools 2397)
It seems impossible to me, but around the time I installed the latest command line tools (xcode-select version 2397) binaries what were built with linkages to @rpath/libfoo.dyld stopped being able to find their dependency. The error looks like this. dyld[1471]: Symbol not found: _GEOSGeomGetX Referenced from: <16DBE67F-CB32-31EE-BCE0-BFB58EEC9740> /Users/pramsey/tmp/capi_indexed_predicate Expected in: <no uuid> unknown zsh: abort ./capi_indexed_predicate If I turn on DYLD_PRINT_SEARCHING, I can see the linker giving up on the rpath entry. dyld[1501]: find path "@rpath/libgeos_c.1.dylib" dyld[1501]: not found: "@rpath/libgeos_c.1.dylib" I can work around, partially, by setting DYLD_LIBRARY_FALLBACK_PATH to have /usr/local/lib in it, but that is only a partial fix, because SIP will strip that variable for any child processes, which means most of my development and database work is borked. This seems like a very new quirk, maybe related to the XCode 15 update, at least in my personal time line, has anyone else seen it, or have any clue as to what has changed? (Worth noting, nothing changed in my code, just one day my builds wouldn't run anymore, and that day was the day after I had installed the new commandline tools).
15
5
4.4k
May ’24
ld: Assertion failed: (resultIndex < sectData.atoms.size())
Hi, I wanted to compile a project (For the compilation I used gcc-13 from home-brew) and I got the following error from the linker: 0 0x100e0f648 __assert_rtn + 72 1 0x100d43fac ld::AtomPlacement::findAtom(unsigned char, unsigned long long, ld::AtomPlacement::AtomLoc const*&, long long&) const + 1204 2 0x100d59924 ld::InputFiles::SliceParser::parseObjectFile(mach_o::Header const*) const + 15164 3 0x100d66e30 ld::InputFiles::parseAllFiles(void (ld::AtomFile const*) block_pointer)::$_7::operator()(unsigned long, ld::FileInfo const&) const + 420 4 0x1a95f0440 _dispatch_client_callout2 + 20 5 0x1a9603f1c _dispatch_apply_invoke + 224 6 0x1a95f0400 _dispatch_client_callout + 20 7 0x1a9601fb8 _dispatch_root_queue_drain + 684 8 0x1a96026c0 _dispatch_worker_thread2 + 164 9 0x1a979c038 _pthread_wqthread + 228 ld: Assertion failed: (resultIndex < sectData.atoms.size()), function findAtom, file Relocations.cpp, line 1336. collect2: error: ld returned 1 exit status I am using a MacBook Pro M2 (Ventura 13.5.2 (22G91)) and the CommandLineTools version 15.0. I have read through some earlier posts about linker issues, which were solved by downgrading Xcode. After downgrading to 14.3.0 I was able to compile the code. Did anyone run into the same Problem? If so, is there a better solution for this? Best regards
25
10
9.4k
Jan ’24
Xcode 15 beta 6: ld: warning: building for 'macCatalyst', but linking in dylib built
In Xcode 15 beta 6, building any Mac Catalyst project will encounter the following Linker warning. ld: warning: building for 'macCatalyst', but linking in dylib (/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa.tbd) built for 'macOS'
23
11
5.8k
Jun ’24
Interface is not propagated when using mergeable libraries
Hi! I've been dealing with mergeable libraries quite some time. However I can't achieve the following scenario: I have 2 xcframeworks A and B that are merged as part of a third xcframework called C. And my app needs to import something from A and B. As per the documentation we have to remove the references from A and B from the final app and replace it with C. If I work on the same xcodepoj it works like a charm (maybe because of caches), but if I try to compile C as a separate XCFramework and distribute it as a packed library, the app is not able to resolve the symbols to A and B classes. This C xcframework is compiled with BUILD_LIBRARY_FOR_DISTRIBUTION se to true and if I check its swiftinterface files it is not declaring the symbols from A and B. However the size of the binary seems to have A and B. Is there any way to export A and B symbols as part of C's swiftinterface? If I add @_exported, it is forcing me to declare the import of A and B wherever I use them and therefore I'm going to have duplicated symbols Thanks!
2
0
861
Jun ’24
Library missing
Second attempt to publish the Mac version of my App, but it was rejected again due to the same error: 'Library missing'. The library IS included with the build. The extracted .app from .xcarchive runs without a problem on different machines, both M1 and Intel. So, I don't know what's wrong. Can somebody please help me? I can provide the complete .ips from Apple if needed. { "code": 1, "flags": 518, "namespace": "DYLD", "indicator": "Library missing", "details": [ "(terminated at launch; ignore backtrace)" ], "reasons": [ "Library not loaded: @rpath/TitaniumKit.framework/Versions/A/TitaniumKit", "Referenced from: &lt;85BA8613-0157-3B28-99AF-E73F1E579B72&gt; /Applications/TiDesigner.app/Contents/MacOS/TiDesigner", "Reason: tried: '/usr/lib/swift/TitaniumKit.framework/Versions/A/TitaniumKit' (no such file, not in dyld cache), '/System/Volumes/Preboot/Cryptexes/OS/usr/lib/swift/TitaniumKit.framework/Versions/A/TitaniumKit' (no such file), '/usr/lib/swift/TitaniumKit.framework/Versions/A/TitaniumKit' (no such file, not in dyld cache), '/System/Volumes/Preboot/Cryptexes/OS/usr/lib/swift/TitaniumKit.framework/Versions/A/TitaniumKit' (no such file), '/System/Library/Frameworks/TitaniumKit.framework/Versions/A/TitaniumKit' (no such file, not in dyld cache), (security policy does not allow @ path expansion)" ] }
4
0
3.5k
Mar ’24
Missing Library Error in Mac Catalyst when distributed through Testflight
The iOS version of the app is functioning correctly on TestFlight, and there are no issues when building the Mac Catalyst version of the app from Xcode. However, when distributing the app to TestFlight for Mac Catalyst, the FFmpegkit library fails to load, resulting in a crash. The crash report indicates that the library is missing and cannot be located in the designated file path. Specifically, the crash report states that the library could not be loaded from "@rpath/ffmpegkit.framework/ffmpegkit". Despite extensive efforts, I have been unable to resolve this issue. This is error message in the crash report. Termination Reason: Namespace DYLD, Code 1 Library missing Library not loaded: @rpath/ffmpegkit.framework/ffmpegkit Referenced from: <9162F8B0-7112-310B-8EDA-59766087927F> /Applications/MyApp.app/Contents/MacOS/MyApp Reason: tried: '/System/Library/Frameworks/ffmpegkit.framework/ffmpegkit' (no such file, not in dyld cache), (security policy does not allow @ path expansion) (terminated at launch; ignore backtrace) Are there any alternative solutions to resolve this problem? This is all the things I have already tried: I tried most of the solutions mentioned in the post below. https://stackoverflow.com/questions/24333981/ios-app-with-framework-crashed-on-device-dyld-library-not-loaded-xcode-6-beta I checked the package content found in Testflight and the paths are correctly matching the following /System/Library/Frameworks/ffmpegkit.framework/ffmpegkit I ran the app scheme as release within Xcode and it build and ran fine with no issues I manually loaded the signing certificates for mac catalyst and still getting the error Deleted all derived data Deleted and reinstalled Xcode and also tried previous Xcode versions. Deleted the project completely
5
1
1.1k
Sep ’24
An Apple Library Primer
Apple’s library technology has a long and glorious history, dating all the way back to the origins of Unix. This does, however, mean that it can be a bit confusing to newcomers. This is my attempt to clarify some terminology. If you have any questions or comments about this, start a new thread and tag it with Linker so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" An Apple Library Primer Apple’s tools support two related concepts: Platform — This is the platform itself; macOS, iOS, iOS Simulator, and Mac Catalyst are all platforms. Architecture — This is a specific CPU architecture used by a platform. arm64 and x86_64 are both architectures. A given architecture might be used by multiple platforms. The most obvious example of this arm64, which is used by all of the platforms listed above. Code built for one platform will not work on another platform, even if both platforms use the same architecture. Code is usually packaged in either a Mach-O file or a static library. Mach-O is used for executables, dynamic libraries, bundles, and object files. These can have a variety of different extensions; the only constant is that .o is always used for a Mach-O containing an object file. Use otool and nm to examine a Mach-O file. Use vtool to quickly determine the platform for which it was built. Use size to get a summary of its size. Use dyld_info to get more details about a dynamic library. IMPORTANT All the tools mentioned here are documented in man pages; for information on how to access that documentation, see Reading UNIX Manual Pages. The term Mach-O image refers to a Mach-O that can be loaded and executed without further processing. That includes executables, dynamic libraries, and bundles, but not object files. A dynamic library has the extension .dylib. You may also see this called a shared library. A framework is a bundle structure with the .framework extension that has both compile-time and run-time roles: At compile time, the framework combines the library’s headers and its stub library (stub libraries are explained below). At run time, the framework combines the library’s code, as a Mach-O dynamic library, and its associated resources. The exact structure of a framework varies by platform. For the details, see Placing Content in a Bundle. macOS supports both frameworks and standalone dynamic libraries. Other Apple platforms support frameworks but not standalone dynamic libraries. Historically these two roles were combined, that is, the framework included the headers, the dynamic library, and its resources. These days Apple ships different frameworks for each role. That is, the macOS SDK includes the compile-time framework and macOS itself includes the run-time one. Most third-party frameworks continue to combine these roles. A static library is an archive of one or more object files. It has the extension .a. Use ar, libtool, and ranlib to inspect and manipulate these archives. The static linker, or just the linker, runs at build time. It combines various inputs into a single output. Typically these inputs are object files, static libraries, dynamic libraries, and various configuration items. The output is most commonly a Mach-O image, although it’s also possible to output an object file. The linker may also output metadata, such as a link map (see Using a Link Map to Track Down a Symbol’s Origin). The linker has seen three major implementations: ld — This dates from the dawn of Mac OS X. ld64 — This was a rewrite started in the 2005 timeframe. Eventually it replaced ld completely. If you type ld, you get ld64. ld_prime — This was introduced with Xcode 15. This isn’t a separate tool. Rather, ld now supports the -ld_classic and -ld_new options to select a specific implementation. Note During the Xcode 15 beta cycle these options were -ld64 and -ld_prime. I continue to use those names because the definition of new changes over time (some of us still think of ld64 as the new linker ;–). The dynamic linker loads Mach-O images at runtime. Its path is /usr/lib/dyld, so it’s often referred to as dyld, dyld, or DYLD. Personally I pronounced that dee-lid, but some folks say di-lid and others say dee-why-el-dee. IMPORTANT Third-party executables must use the standard dynamic linker. Other Unix-y platforms support the notion of a statically linked executable, one that makes system calls directly. This is not supported on Apple platforms. Apple platforms provide binary compatibility via system dynamic libraries and frameworks, not at the system call level. Note Apple platforms have vestigial support for custom dynamic linkers (your executable tells the system which dynamic linker to use via the LC_LOAD_DYLINKER load command). This facility originated on macOS’s ancestor platform and has never been a supported option on any Apple platform. The dynamic linker has seen 4 major revisions. See WWDC 2017 Session 413 (referenced below) for a discussion of versions 1 through 3. Version 4 is basically a merging of versions 2 and 3. The dyld man page is chock-full of useful info, including a discussion of how it finds images at runtime. One of the most common points of confusion with dynamic linker is the way that the dynamic linker identifies dynamic libraries. There are two standard approaches to this, as described in Dynamic Library Identification. Mach-O images are position independent, that is, they can be loaded at any location within the process’s address space. Historically, Mach-O supported the concept of position-dependent images, ones that could only be loaded at a specific address. While it may still be possible to create such an image, it’s no longer a good life choice. Mach-O images have a default load address, also known as the base address. For modern position-independent images this is 0 for library images and 4 GiB for executables (leaving the bottom 32 bits of the process’s address space unmapped). When the dynamic linker loads an image, it chooses an address for the image and then rebases the image to that address. If you take that address and subtract the image’s load address, you get a value known as the slide. Xcode 15 introduced the concept of a mergeable library. This a dynamic library with extra metadata that allows the linker to embed it into the output Mach-O image, much like a static library. Mergeable libraries have many benefits. For all the backstory, see WWDC 2023 Session 10268 Meet mergeable libraries. For instructions on how to set this up, see Configuring your project to use mergeable libraries. If you put a mergeable library into a framework structure you get a mergeable framework. Xcode 15 also introduced the concept of a static framework. This is a framework structure where the framework’s dynamic library is replaced by a static library. Note It’s not clear to me whether this offers any benefit over creating a mergeable framework. Earlier versions of Xcode did not have proper static framework support. That didn’t stop folks trying to use them, which caused all sorts of weird build problems. A universal binary is a file that contains multiple architectures for the same platform. Universal binaries always use the universal binary format. Use the file command to learn what architectures are within a universal binary. Use the lipo command to manipulate universal binaries. A universal binary’s architectures are either all in Mach-O format or all in the static library archive format. The latter is called a universal static library. A universal binary has the same extension as its non-universal equivalent. That means a .a file might be a static library or a universal static library. Most tools work on a single architecture within a universal binary. They default to the architecture of the current machine. To override this, pass the architecture in using a command-line option, typically -arch or --arch. An XCFramework is a single document package that includes libraries for any combination of platforms and architectures. It has the extension .xcframework. An XCFramework holds either a framework, a dynamic library, or a static library. All the elements must be the same type. Use xcodebuild to create an XCFramework. For specific instructions, see Xcode Help > Distribute binary frameworks > Create an XCFramework. Historically there was no need to code sign libraries in SDKs. If you shipped an SDK to another developer, they were responsible for re-signing all the code as part of their distribution process. Xcode 15 changes this. You should sign your SDK so that a developer using it can verify this dependency. For more details, see WWDC 2023 Session 10061 Verify app dependencies with digital signatures and Verifying the origin of your XCFrameworks. A stub library is a compact description of the contents of a dynamic library. It has the extension .tbd, which stands for text-based description (TBD). Apple’s SDKs include stub libraries to minimise their size; for the backstory, read this post. Stub libraries currently use YAML format, a fact that’s relevant when you try to interpret linker errors. Use the tapi tool to create and manipulate these files. In this context TAPI stands for a text-based API, an alternative name for TBD. Oh, and on the subject of tapi, I’d be remiss if I didn’t mention tapi-analyze! Historically, the system maintained a dynamic linker shared cache, built at runtime from its working set of dynamic libraries. In macOS 11 and later this cache is included in the OS itself. Libraries in the cache are no longer present in their original locations on disk: % ls -lh /usr/lib/libSystem.B.dylib ls: /usr/lib/libSystem.B.dylib: No such file or directory Apple APIs, most notably dlopen, understand this and do the right thing if you supply the path of a library that moved into the cache. That’s true for some, but not all, command-line tools, for example: % dyld_info -exports /usr/lib/libSystem.B.dylib /usr/lib/libSystem.B.dylib [arm64e]: -exports: offset symbol … 0x5B827FE8 _mach_init_routine % nm /usr/lib/libSystem.B.dylib …/nm: error: /usr/lib/libSystem.B.dylib: No such file or directory Mach-O uses a two-level namespace. When a Mach-O image imports a symbol, it references the symbol name and the library where it expects to find that symbol. This improves both performance and reliability but it precludes certain techniques that might work on other platforms. For example, you can’t define a function called printf and expect it to ‘see’ calls from other dynamic libraries because those libraries import the version of printf from libSystem. To help folks who rely on techniques like this, macOS supports a flat namespace compatibility mode. This has numerous sharp edges — for an example, see the posts on this thread — and it’s best to avoid it where you can. If you’re enabling the flat namespace as part of a developer tool, search the ’net for dyld interpose to learn about an alternative technique. WARNING Dynamic linker interposing is not documented as API. While it’s a useful technique for developer tools, do not use it in products you ship to end users. Apple platforms use DWARF. When you compile a file, the compiler puts the debug info into the resulting object file. When you link a set of object files into a executable, dynamic library, or bundle for distribution, the linker does not include this debug info. Rather, debug info is stored in a separate debug symbols document package. This has the extension .dSYM and is created using dsymutil. Use symbols to learn about the symbols in a file. Use dwarfdump to get detailed information about DWARF debug info. Use atos to map an address to its corresponding symbol name. Different languages use different name mangling schemes: C, and all later languages, add a leading underscore (_) to distinguish their symbols from assembly language symbols. C++ uses a complex name mangling scheme. Use the c++filt tool to undo this mangling. Likewise, for Swift. Use swift demangle to undo this mangling. Over the years there have been some really good talks about linking and libraries at WWDC, including: WWDC 2023 Session 10268 Meet mergeable libraries WWDC 2022 Session 110362 Link fast: Improve build and launch times WWDC 2022 Session 110370 Debug Swift debugging with LLDB WWDC 2021 Session 10211 Symbolication: Beyond the basics WWDC 2019 Session 416 Binary Frameworks in Swift — Despite the name, this covers XCFrameworks in depth. WWDC 2018 Session 415 Behind the Scenes of the Xcode Build Process WWDC 2017 Session 413 App Startup Time: Past, Present, and Future WWDC 2016 Session 406 Optimizing App Startup Time Note The older talks are no longer available from Apple, but you may be able to find transcripts out there on the ’net. Historically Apple published a document, Mac OS X ABI Mach-O File Format Reference, or some variant thereof, that acted as the definitive reference to the Mach-O file format. This document is no longer available from Apple. If you’re doing serious work with Mach-O, I recommend that you find an old copy. It’s definitely out of date, but there’s no better place to get a high-level introduction to the concepts. The Mach-O Wikipedia page has a link to an archived version of the document. For the most up-to-date information about Mach-O, see the declarations and doc comments in <mach-o/loader.h>. Revision History 2024-10-07 Added some basic information about the dynamic linker shared cache. 2024-07-26 Clarified the description of the expected load address for Mach-O images. 2024-07-23 Added a discussion of position-independent images and the image slide. 2024-05-08 Added links to the demangling tools. 2024-04-30 Clarified the requirement to use the standard dynamic linker. 2024-03-02 Updated the discussion of static frameworks to account for Xcode 15 changes. Removed the link to WWDC 2018 Session 415 because it no longer works )-: 2024-03-01 Added the WWDC 2023 session to the list of sessions to make it easier to find. Added a reference to Using a Link Map to Track Down a Symbol’s Origin. Made other minor editorial changes. 2023-09-20 Added a link to Dynamic Library Identification. Updated the names for the static linker implementations (-ld_prime is no more!). Removed the beta epithet from Xcode 15. 2023-06-13 Defined the term Mach-O image. Added sections for both the static and dynamic linkers. Described the two big new features in Xcode 15: mergeable libraries and dependency verification. 2023-06-01 Add a reference to tapi-analyze. 2023-05-29 Added a discussion of the two-level namespace. 2023-04-27 Added a mention of the size tool. 2023-01-23 Explained the compile-time and run-time roles of a framework. Made other minor editorial changes. 2022-11-17 Added an explanation of TAPI. 2022-10-12 Added links to Mach-O documentation. 2022-09-29 Added info about .dSYM files. Added a few more links to WWDC sessions. 2022-09-21 First posted.
0
0
7.3k
Oct ’24
dyld: Symbol not found: swift34swift50override_conformsToProtocol
I am getting following error from one of the pod frameworks while running the app (Build is a success). dyld: Symbol not found: __ZN5swift34swift50override_conformsToProtocolEPKNS_14TargetMetadataINS_9InProcessEEEPKNS_24TargetProtocolDescriptorIS1_EEPFPKNS_18TargetWitnessTableIS1_EES4_S8_E. Referenced from: X framework Expected in: frameworks/DeviceKit.framework/DeviceKit mac OS 10.15 Xcode 12.4 React native 0.63 cocoapods: 1.10.1
9
2
14k
1w