dlopen not using pre-loaded dependencies (MacOS)

Question

Hi,

A cross-platform plugin architecture we developed needs to load libraries at runtime from arbitrary locations (within our plugins).

On MacOS, dynamic library loading fails to find/use a dependency that was already loaded, despite the library "install-name" matching. The same technique works on Linux for a library "SONAME" and on Windows based on the DLL filename, however, on Mac, it seems like the dependency is not resolved, unless it is also on a path of library locations (eg. DYLD_LIBRARY_PATH).

Am I missing the proper technique to achieve the desired behavior? Is this the expected behavior on Mac or an issue in dlopen resolving dependencies?

For example:

dlopen(/Users/craig/KayakSDK/Plugins/ca.digitalrapids.CommonMedia/bin/OS_X/libCommonMedia.dylib, 1): Library not loaded: libKayakNative.dylib

Referenced from: /Users/craig/KayakSDK/Plugins/ca.digitalrapids.CommonMedia/bin/OS_X/libCommonMedia.dylib

Reason: image not found

But libKayakNative.dylib had already been loaded and has the expected "install-name".

$ otool -D Plugins/ca.digitalrapids.KayakCore/bin/OS_X/libKayakNative.dylib

Plugins/ca.digitalrapids.KayakCore/bin/OS_X/libKayakNative.dylib:

libKayakNative.dylib

So why doesn't dlopen utilize the already loaded libKayakNative.dylib ? That's how SONAME on Linux works and the DLL name on Windows.

If DYLD_LIBRARY_PATH specifies the folders within the plugins, then everything does load and execute fine. But this is far from ideal, as DYLD_LIBRARY_PATH would need to be configured ahead of launch, and cannot be modified at runtime by the application. If there was a way to modify DYLD_LIBRARY_PATH (or an equivalent) at runtime, that would work for us too.

Thanks, any info is appreciated,

Craig

XPC

9.4k

Posted by

cwhite102

Reply

Add a Comment

Answer 1

I had a somewhat similar problem, which I was able to solve by adding this to my Info.plist:

    <key>LSEnvironment</key>
    <dict>
      <key>DYLD_LIBRARY_PATH</key>
      <string>@executable_path/../Resources/lib</string>
    </dict>

When the app starts up, the environment contains DYLD_LIBRARY_PATH=@executable_path/../Resources/lib (verbatim, exactly as shown), but the dlopen calls work. Without this (or other tricks like having the app add DYLD_LIBRARY_PATH to the environment and then exec itself), the dlopen calls fail.

@cwhite102, I wonder if you could solve your somewhat more complex problem by putting all possible dylib paths in the environment (perhaps via Info.plist, as described above), and then always using a full absolute path (or maybe a path explicitly using @executable_path, if that works?) with dlopen. Perhaps dyld would then be satisfied that the dylib was coming from a path that it knew about, and would load the exact file you specified. Of course, if you need to load dylibs from outside the app install location, then that doesn't help. In that case, you might need to have the app set DYLD_LIBRARY_PATH and then exec itself.

Posted by

ChrisMacGregor

It turns out that what I describe above doesn't work (and I can't seem to edit or delete my answer?), at least not on recent macOS with SIP enabled. Regardless of how you try to do it (setenv/putenv followed by exec, or via LSEnvironment), all environment variables starting with DYLD_ are ignored (scrubbed out of the environment, in fact).

I haven't tested it, because then I also found that it was working for me even without setting DYLD_LIBRARY_PATH (perhaps I was missing a library that needed to be patched to use @executable_path? unclear), but I believe this entitlement will allow the use of the DYLD_* environment variables:

https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-dyld-environment-variables?language=objc

—
ChrisMacGregor

Add a Comment

dlopen not using pre-loaded dependencies (MacOS)

Replies