Post

Replies

Boosts

Views

Activity

Reply to dlopen taking huge amount of time in macOS Mojave and Catalina
Hi, we're experiencing the same issue here running Catalina 10.15.7, even when internet connectivity is off. We saw that during the 30 seconds delay the process CPU has been at 100%, and when we ran a few core dumps during the 30 second delay we found several stack traces of Apple's ImageLoader code running for the entire duration of the 30 second hang. example backtrace (most of them were stuck in trieWalk): 	* frame #0: 0x0000000110e15889 dyld`ImageLoader::trieWalk(unsigned char const*, unsigned char const*, char const*) + 177 		frame #1: 0x0000000110e1f400 dyld`ImageLoaderMachOCompressed::findShallowExportedSymbol(char const*, ImageLoader const**) const + 112 		frame #2: 0x0000000110e19075 dyld`ImageLoaderMachO::findExportedSymbol(char const*, bool, char const*, ImageLoader const**) const + 37 		frame #3: 0x0000000110e145c3 dyld`ImageLoader::weakBindOld(ImageLoader::LinkContext const&) + 1485 		frame #4: 0x0000000110e1228f dyld`ImageLoader::link(ImageLoader::LinkContext const&, bool, bool, bool, ImageLoader::RPathChain const&, char const*) + 333 		frame #5: 0x0000000110e04a01 dyld`dyld::link(ImageLoader*, bool, bool, ImageLoader::RPathChain const&, unsigned int) + 161 		frame #6: 0x0000000110e0ee0b dyld`dlopen_internal + 477
Nov ’20
Reply to dlopen taking huge amount of time in macOS Mojave and Catalina
After further investigation we've found that all of the symbols from our .so files are exported. About 2/3 of them are bounded successfully by ImageLoader::recursiveBind() almost instantaneously, but the last 1/3 is handled by ImageLoader::weakBindOld(), it searches for them, fails to "strong bind" them and logs them as "found weak". I'm still not sure what's the difference between regular binding and a weak binding, so some context on what we should change in our build process would really help to resolve this.
Nov ’20
Reply to dlopen taking huge amount of time in macOS Mojave and Catalina
From what I read in the ImageLoader::weakBindOld - https://opensource.apple.com/source/dyld/dyld-750.6/src/ImageLoader.cpp.auto.html it looks like that this is the process of binding undefined symbols in one library with their implementation dependent libraries. The function is going through all the images that were just loaded that need coalescing, goes through every undefined symbol and iterates through all other loaded images to find the implementation for it using a trie search according to the symbol's name. We compile these .so/.dylib files for Linux and Mac, and it seems that Linux does this binding process in less that 1 second. Where can I get some guidance to figure out what should we try to change in our compilation/linkage process to mitigate this?
Nov ’20
Reply to dlopen taking huge amount of time in macOS Mojave and Catalina
Yes, these libraries are C++ libraries, they include libraries that have a huge amount of symbols like boost and folly, and since everything is built into dynamic libraries a lot of load time coalescing is expected. What bothers me is the Apple specific implementation in dyld. Our code is fully portable and this issue doesn't happen on Linux, even though it's about the same amount of symbols. This looks like a bug in weakBind.
Nov ’20