Just as an update, the Homebrew issue thread above helped narrow down the optimization pass for Vim using a bisect. If we build it with -mllvm -enable-constraint-elimination=0 it will remove the issue. From looking at what constraint elimination does that sounds like it would likely be the same issue as JDK where the compiler misunderstands the constraints. I'm playing around with this flag a little more to see how it behaves.

[quote='810521022, y-c-c, /thread/766718?answerId=810521022#810521022, /profile/y-c-c'] I did force functions to be inlined and that removes the problem. I personally think the inlining is just exposing the problem though, as multiple inlined functions allowed the optimizer to optimize more. [/quote] Small correction: I meant I forced functions to not be inlined and that removes the problem.

[quote='810430022, endecotp, /thread/766718?answerId=810430022#810430022, /profile/endecotp'] Have either of you tried compiling with more fine-grained optimisation settings? In particular, have you tried with -O3 but with vectorisation disabled? [/quote] Just tried to turn off vectorization and the bug still occurs for me. Looking at the disassembly I don't think there's any weird vectorization going on. I did force functions to be inlined and that removes the problem. I personally think the inlining is just exposing the problem though, as multiple inlined functions allowed the optimizer to optimize more.

I couldn't add a comment to that JDK thread (is there no way to register an account?) but I posted about another Xcode clang miscompilation bug about a week ago as well (https://developer.apple.com/forums/thread/766030). It doesn't look identical to what the JDK bug is reporting but there are some similarities, including wrong codegen that leads to subtle offset when looping, and only happening in -O2/-O3. This codegen bug is currently blocking Vim from being upgraded to using the latest toolchain for CI, and is actually causing the active distributed Vim in Homebrew to be miscompiled, because Homebrew believes in using the latest Apple toolchains (which Apple encourages). Edit: I just tested, and the system default Vim in macOS 15 (/usr/bin/vim) also shows the same bug. The repro steps is at https://github.com/Homebrew/homebrew-core/issues/195325 I share the above commenter's frustration that while Apple tells us to use the Feedback Assistant to file bugs, it is essentially a black hole with zero feedbacks. I still have bugs that I opened years ago that remain open. Miscompilation bugs seem like a pretty serious class of bugs in compilers. It's one thing for clang to crash or fail to compile or suffer performance regressions. It's a completely different issue if it generates subtly incorrect code which has all sorts of ramifications as compilers are pretty low-level (hence relied on by a lot of software) in the software supply chain, and need to be trusted to generate correct code. Is it possible to get some more info than "no info to share"? As I mentioned I think miscompilation bugs are serious enough that developers deserve to know if they should be warned to not upgrade to the latest toolchain if their software are going to break in subtle ways. JDK is lucky that this is immediately throwing a runtime exception that it got noticed.

Thanks for the response. I ended up filing a bug anyway (FB15489959).

Right. Thanks a lot for the additional pointers! I didn't know about pathconf. But you are right, this does sound like a pain. It's a good thing to know this in my backpocket but for this particular issue (fixing a problem in a popular open-source tool) I will aim for a more conservative approach (allocate at least enough memory and be conservative in the file name size when creating a derived file name). But yeah I think you didn't even point out all the complexities in the file name validation issues since this is such a surprisingly complicated topic. HFS+ automatically normalize all UTF-8 code points, while APFS preserves existing UTF-8 code points without normalization (which I remember caused some teeth gnashing). It does seem a little unsatisfactory to me because the "create a file and see if it fails" strategy is a reactive one and prevents you from doing certain things (e.g. properly picking the correct file name length, or pre-processing to safely escape file name characters) but given the complexity of the file name validation rules it may not be something that's easy to expose (not to mention that file name truncation is itself something to worry about if you don't want to cut off in the middle of a multi-byte-UTF-8 character).

@eskimo I'm debugging some code in a cross-platform C app that is doing some wildcard substitution and manually allocating enough space to hold file names when concatenating file names together. Admittedly this is probably not the best way to do it and ideally we do dynamic allocation only after knowing how long of a name we have, and the code also needs some bounds checking to make sure it won't crash. But basically in this situation, we need to know the max file name size in order to do a static allocation. From reading glibc at least (which Apple doesn't use) it seems that their limits are essentially suggestive and not guaranteed to be respected: https://www.gnu.org/software/libc/manual/html_node/Limits-for-Files.html. Anyway for the code I was looking at, it actually is calling readdir(). It's only the concatenation later that causes problems. If you do man dirent or look at usr/include/dirent.h the docs actually does say it's using 1024 bytes for a file name (there isn't a real constant you can use though), same as max path size, so that's probably what I will use for now, although the code probably should be refactored to not have to have to rely on such constants. Edit: Just to add. Another reason why we are interested in the max file name is that we want to be able to append a postfix to the file name. So imagine we have a file some_long_name.txt, and we want to save a backup version called some_long_name.txt.backup. It's useful to be able to make the name as long as possible without truncation, but if the constant is wrong we could end up making too long of a name which fails the system call.

Thanks! I did see the file and saw the 10-bit length but somehow didn’t register. I guess that could be as good as we get as a hard limit as it does mean you can’t get more than 1024 bytes which would match the assertion of it being 255 4-byte characters (since you have to leave one out for the null character). I guess it’s still unclear to me why macOS treats wide 4-byte characters as two chars in the file size limit but I’m wondering if that’s more for backwards compatibility reasons as it doesn’t want you to be able to create files with names that can’t fit in HFS+. I’ll probably just be safe and reserve 1024 bytes.

FWIW seems like HFS+ is 255 UTF-16 characters, as specified here: https://developer.apple.com/library/archive/technotes/tn/tn1150.html I still can't look up the official docs for APFS though. Experimentation suggests that the current macOS behavior is the same: 255 UTF-16 chars, but I don't know if it's the limitation of the tool or a system API. Characters like 'a' (1 byte) and '不' (2 byte) would could as 1 character, and wider characters like 🚧 would be considered 2 characters (so you can only have 127 of them plus another character). I still can't find out where in the documentation where that is stated though as APFS docs state that all file names are UTF-8. I guess I could just be safe and always allocate 255 * 4 assuming that a full 4-byte character could count as only 1 char. I would love it if there's an official docs that I could look up though.

I have the same issue with rgrant@mac.com! Tried to update to Beta 4 (from beta 2 I think) and it bricked my device. Tried to use DFU / Apple Configurator to restore with a freshly download ipsw image of Big Sur Beta 4 and was told "The OS cannot be restored on this device" and it didn't do anything. When I boot up my DTK now it will just boot up, shows nothing on screen, then reboots itself after a couple seconds… Made another thread on this issue: https://developer.apple.com/forums/thread/656780