dlopen not using pre-loaded dependencies (MacOS)

Question

cwhite102 OP

Created May ’20

Replies 16

Boosts 0

Views 11k

Participants 6

Hi,

A cross-platform plugin architecture we developed needs to load libraries at runtime from arbitrary locations (within our plugins).

On MacOS, dynamic library loading fails to find/use a dependency that was already loaded, despite the library "install-name" matching. The same technique works on Linux for a library "SONAME" and on Windows based on the DLL filename, however, on Mac, it seems like the dependency is not resolved, unless it is also on a path of library locations (eg. DYLD_LIBRARY_PATH).

Am I missing the proper technique to achieve the desired behavior? Is this the expected behavior on Mac or an issue in dlopen resolving dependencies?

For example:

dlopen(/Users/craig/KayakSDK/Plugins/ca.digitalrapids.CommonMedia/bin/OS_X/libCommonMedia.dylib, 1): Library not loaded: libKayakNative.dylib

Referenced from: /Users/craig/KayakSDK/Plugins/ca.digitalrapids.CommonMedia/bin/OS_X/libCommonMedia.dylib

Reason: image not found

But libKayakNative.dylib had already been loaded and has the expected "install-name".

$ otool -D Plugins/ca.digitalrapids.KayakCore/bin/OS_X/libKayakNative.dylib

Plugins/ca.digitalrapids.KayakCore/bin/OS_X/libKayakNative.dylib:

libKayakNative.dylib

So why doesn't dlopen utilize the already loaded libKayakNative.dylib ? That's how SONAME on Linux works and the DLL name on Windows.

If DYLD_LIBRARY_PATH specifies the folders within the plugins, then everything does load and execute fine. But this is far from ideal, as DYLD_LIBRARY_PATH would need to be configured ahead of launch, and cannot be modified at runtime by the application. If there was a way to modify DYLD_LIBRARY_PATH (or an equivalent) at runtime, that would work for us too.

Thanks, any info is appreciated,

Craig

Boost

Answer 1

ChrisMacGregor OP

Feb ’22

I had a somewhat similar problem, which I was able to solve by adding this to my Info.plist:

    <key>LSEnvironment</key>
    <dict>
      <key>DYLD_LIBRARY_PATH</key>
      <string>@executable_path/../Resources/lib</string>
    </dict>

When the app starts up, the environment contains DYLD_LIBRARY_PATH=@executable_path/../Resources/lib (verbatim, exactly as shown), but the dlopen calls work. Without this (or other tricks like having the app add DYLD_LIBRARY_PATH to the environment and then exec itself), the dlopen calls fail.

@cwhite102, I wonder if you could solve your somewhat more complex problem by putting all possible dylib paths in the environment (perhaps via Info.plist, as described above), and then always using a full absolute path (or maybe a path explicitly using @executable_path, if that works?) with dlopen. Perhaps dyld would then be satisfied that the dylib was coming from a path that it knew about, and would load the exact file you specified. Of course, if you need to load dylibs from outside the app install location, then that doesn't help. In that case, you might need to have the app set DYLD_LIBRARY_PATH and then exec itself.

0