8 Replies
      Latest reply on Sep 13, 2019 3:10 AM by tguillem
      jedisct1 Level 1 Level 1 (0 points)

        There seems to be a regression in Xcode 11 beta, when AVX is enabled. The C library doesn't make the same assertions regarding alignment as the compiler.

         

        Here is a C-reduced test case:

         

        #include 
        #include 
        #include 
        #include 
        #include 
        
        typedef struct {
            char  d[16];
            void *e;
            struct { char b[5536]; } f;
        } i;
        
        void g(void) {
            struct addrinfo hints, *k;
            memset(&hints, 0, sizeof hints);
            getaddrinfo(NULL, NULL, &hints, &k);
        }
        
        int main(void) {
            puts("Hello world");
            fflush(stdout);
            close(open("/dev/null", O_RDONLY));
            i context;
            context.e = open;
            printf("%p\n", context.d);
            g();
            return 0;
        }

         

        When compiled with AVX optimizations (e.g. -mavx or more commonly -march=native), this crashes even before `main()` is executed:

         

        cc -mavx -O2 a.c && ./a.out
        lldb ./a.out
        run
          * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
        frame #0: 0x00007fff65e73316 libdyld.dylib`stack_not_16_byte_aligned_error libdyld.dylib`stack_not_16_byte_aligned_error:
        ->  0x7fff65e73316 <+0>: movdqa %xmm0, (%rsp)

         

        Without AVX optimizations, or with Xcode 10, this doesn't happen.

         

        This bug affects real-world applications such as libsodium and dsvpn.

         

        A workaround is to use `-ffreestanding`.

        • Re: Xcode 11b6: stack_not_16_byte_aligned_error with AVX code
          jedisct1 Level 1 Level 1 (0 points)

          The headers got mangled. Not the most important thing, but anyway, here they are:

           

          #include <fcntl.h>
          #include <netdb.h>
          #include <stdio.h>
          #include <string.h>
          #include <unistd.h>
          
          • Re: Xcode 11b6: stack_not_16_byte_aligned_error with AVX code
            snowcat Level 1 Level 1 (0 points)

            Unfortunate combination:

            • -mavx
            • -O2
            • -fstack-check (enabled by default when building for 10.15)

             

            Here's the start of main with all these options:

            a.out[0x100000e34] <+4>:   pushq  %rbx
            a.out[0x100000e35] <+5>:   andq   $-0x20, %rsp
            a.out[0x100000e39] <+9>:   pushq  %rax
            a.out[0x100000e3a] <+10>:  movl   $0x1640, %eax             ; imm = 0x1640 
            a.out[0x100000e3f] <+15>:  callq  0x100000ef6               ; symbol stub for: ___chkstk_darwin
            a.out[0x100000e44] <+20>:  subq   %rax, %rsp
            a.out[0x100000e47] <+23>:  popq   %rax

             

            The andq is only there with -mavx and -O2. This aligns the stack to 16.

             

            The call to ___chkstk_darwin is only there with the stack check feature enabled. For specifically this call, the stack is aligned to 8 if it was previously aligned to 16. This is where the error occurs.

             

            Side note: I'm not super confident in how much this stuff is being tested. I filed a bug a few weeks ago that your test case actually also triggers. If you build like this you get a different crash:

             

               cc -all_load a.c && ./a.out

             

            (This links in the compiler runtime that ships with Xcode, which has a different ___chkstk_darwin that modifies rax, and you can see that the code I quoted above doesn't expect that.)

            • Re: Xcode 11 GM: stack_not_16_byte_aligned_error with AVX code
              jedisct1 Level 1 Level 1 (0 points)

              This critical bug is still present in XCode 11 GM.

                • Re: Xcode 11 GM: stack_not_16_byte_aligned_error with AVX code
                  jcookie Level 2 Level 2 (70 points)

                  I have reported this issue as early as 8. August, with no reply from Apple. I am a bit alarmed that a bug as critical as this has been seemingly ignored for over a month. This essentially makes C/C++ compeltely useless for any non-trivial numerical code.

                    • Re: Xcode 11 GM: stack_not_16_byte_aligned_error with AVX code
                      tguillem Level 1 Level 1 (0 points)

                      Hello, I don't know if it's related but a simple

                       

                      void *buf = aligned_alloc(32, 128);

                      cause a SIGABORT with the following stack strace.

                       

                      * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT

                          frame #0: 0x0000000104a22418 dyld`__abort_with_payload + 8

                          frame #1: 0x0000000104a21a58 dyld`abort_with_payload_wrapper_internal + 100

                          frame #2: 0x0000000104a21a88 dyld`abort_with_payload + 12

                          frame #3: 0x00000001049e48f8 dyld`dyld::halt(char const*) + 304

                          frame #4: 0x00000001049e4a14 dyld`dyld::fastBindLazySymbol(ImageLoader**, unsigned long) + 284

                          frame #5: 0x00000001db73f848 libdyld.dylib`dyld_stub_binder + 60

                        * frame #6: 0x000000010490e1fc alignedalloc`main(argc=1, argv=0x000000016b4f78d0) at main.m:13:20

                          frame #7: 0x00000001db73d8e0 libdyld.dylib`start + 4

                      • Re: Xcode 11 GM: stack_not_16_byte_aligned_error with AVX code
                        snowcat Level 1 Level 1 (0 points)

                        You're not wrong, but there are workarounds in case you (or someone else finding this thread) just really need to make it work

                         

                        • add -fno-stack-check to CFLAGS/CXXFLAGS, or
                        • set MACOSX_DEPLOYMENT_TARGET=10.14 in the environment or choose 10.14 as deployment target in Xcode
                          • Re: Xcode 11 GM: stack_not_16_byte_aligned_error with AVX code
                            jcookie Level 2 Level 2 (70 points)

                            True, and I understand that beta software can have bugs that require workarounds like this.

                             

                            But we are talkign about Xcode GM now. Imagine the mess if this bug is present in the official Catalina release. For instance, we have academic personell (non-programmers) who heavily rely on open-source numerical software, and its a common practice to build this from source using homebrew (to make sure that latest CPU capabilities are properly taken care of). These people wouldn't know why their software is randomly crashign and they don't understand compiler flags or how to look for solutions. All they know is how to run "brew install --build-from-source X" and they expect it to work.