12766 Views 17 Replies
      Latest reply: Nov 16, 2015 3:51 AM by Ben Hirschberg RSS
      Colin Cornaby Level 1 Level 1 (0 points)
      Colin Cornaby Jun 9, 2015 1:02 PM

        I work for a third party library vendor and we are pretty sure our customers will ask us for a Bitcode version of our library.

         

        We are very performance sensitive though, and we've optimized critical sections of our library in NEON assembly. When we compile to LLVM bitcode, we are assuming that NEON assembly will be encapsulated. Is that correct?

         

        We're also worried about writing code in assembly though. Bitcode seems to leave open the possibility that our application could end up running on a platform who's capabilities we did not optimize for in advance. It seems like it might be a good idea to make sure our software paths are working well. Would it also be a good idea to attempt to drop out of NEON directly and use the LLVM SIMD intrinsics? I believe Accelerate was also tried and found to be too high level for our stuff. We're also multiplatform.

        • Tags:
        • Re: Bitcode and Assembly?
          ChasingPtr Apple Staff Apple Staff (0 points)
          ChasingPtr Jun 12, 2015 10:20 AM (in response to Colin Cornaby)

          Hi Collin,

           

          For iOS, this should *just work*. When you are producing your library, the compiler and linker will include the bitcode in the Mach-O file. There are only two things that you should know.

           

          1. If you are not using Xcode, please manually add -femit-bitcode command line option.

          2. When using Xcode, bitcode is only produced during archive build, not debug or release build.

           

          Evan

            • Re: Bitcode and Assembly?
              bwilson Apple Staff Apple Staff (20 points)
              bwilson Jun 12, 2015 11:06 AM (in response to ChasingPtr)

              One small correction: The option name to include bitcode is "-fembed-bitcode".

               

              As far as the question about whether Neon intrinsics are preferable to writing assembly code, I will say "yes", at least in most cases. If you use intrinsics, the compiler can optimize the code to run well on different processors, and it is generally easier to maintain C code than assembly. But, if you need to hand-optimize the assembly to get the performance you want, you can certainly do that and it should "just work" for iOS even when you have bitcode for the other parts of your code.

                • Re: Bitcode and Assembly?
                  Colin Cornaby Level 1 Level 1 (0 points)
                  Colin Cornaby Jun 12, 2015 1:11 PM (in response to bwilson)

                  Thanks!

                   

                  If we include a non-NEON accelerated path, is there any way to expose that to the Bitcode representation as well in case our code is post-processed for an architecture where NEON isn't present? Or should we not be concerned about that? We're using #defines right now to mark out our NEON path, which I'm assuming would mean only our accelerated section would ever make it into the Bitcode representation.

                • Re: Bitcode and Assembly?
                  dshirley Level 1 Level 1 (0 points)
                  dshirley Jun 15, 2015 5:45 PM (in response to ChasingPtr)

                  > For iOS, this should *just work*.

                   

                  Unfortunately it does not work.

                   

                  I created a library project that contains a very basic .c file and a .s file.

                  The .c file gets compiled into a .o file that does contain a bitcode section; the .s file generates a .o file that does not contain bitcode.

                  The resulting .a also does not contain a bitcode segment.

                   

                  I'm testing if bitcode is added by running "otool -l | grep bitcode"

                   

                  % otool -l Cfile.o | grep bit

                    sectname __bitcode

                   

                  Xcode asks LLVM to use "-fembed-bitcode-marker".  I manually tried using "-femit-bitcode", but with a "unknown argument: '-femit-bitcode'"

                    • Re: Bitcode and Assembly?
                      bwilson Apple Staff Apple Staff (20 points)
                      bwilson Jun 16, 2015 9:04 AM (in response to dshirley)

                      Assembly source files should work in the sense that you can build and run your project. I did not mean to imply that we somehow de-compile the assembly source into bitcode.

                       

                      Searching for a "bitcode" section is not a reliable way to detect if your files contain embedded bitcode. If you want to do that, search for the "__LLVM" segment. You should be aware that a normal build with the -fembed-bitcode-marker option will produce minimal size embedded bitcode sections without any real content. This is done as a way of testing the bitcode-related aspects of your build without slowing down the build process. The actual bitcode content is included when you do an Archive build.

                        • Re: Bitcode and Assembly?
                          dshirley Level 1 Level 1 (0 points)
                          dshirley Jun 16, 2015 11:55 AM (in response to bwilson)

                          Great info @bwilson.  Thank you.

                           

                          > If you want to do that, search for the "__LLVM" segment

                           

                          Ahh, you're correct.  When I searched for the __LLVM segment, I do see that the assembly's .o file contains an __LLVM segment.

                           

                          > Assembly source files should work in the sense that you can build and run your project. I did not mean to imply that we somehow de-compile the assembly source into bitcode.

                           

                          When it becomes a requirement to submit apps in bitcode format, how will this impact architecture specific code (ie. assembly, or anything that is ifdef'd for that matter).  It makes sense that assembly isn't converted to bitcode, but doesn't everything need to be in bitcode in order for an archive to be fully encoded in bitcode?  I have an app that's hitting a compile warning when archiving complaining that a specific 3rd party library doesn't contain bitcode so the app cannot be archived with bitcode.  That 3rd party library won't emit bitcode ostensibly because it contains assembly (I could be wong about the cause, though).

                           

                          > The actual bitcode content is included when you do an Archive build.

                           

                          Does bitcode content get included when static libraries are built?  Dynamic libraries?  If not, how should 3rd party library vendors distribute their libraries such that app developers can include them and then spit out bitcode in an archive?

                           

                          > You should be aware that a normal build with the -fembed-bitcode-marker option will produce minimal size embedded bitcode sections without any real content.

                           

                          Hmm, that's interesting.  I didn't come up with that option myself, that's what Xcode showed in the build logs when I build a static library with bitcode turned on.  I would have thought it would contain "real content", otherwise how will 3rd party libraries get distributed to apps?

                           

                           

                          Thanks again for your responses.  This is a really big change to the way things work and the documentation is quite sparse at the moment.  I really appreciate the help here!

                            • Re: Bitcode and Assembly?
                              dshirley Level 1 Level 1 (0 points)
                              dshirley Jun 17, 2015 9:17 AM (in response to dshirley)

                              One update.  It appears the library project I have was dropping bitcode because it had GENERATE_MASTER_OBJECT_FILE turned on.  This seems like a bug in Xcode.  I filed bug report 21422164.

                               

                              I'm still curious if/how assembly will be packaged in an app archive when it is submitted to iTunes Connect.

                               

                              Thanks!

                               

                              -David

                                • Re: Bitcode and Assembly?
                                  bwilson Apple Staff Apple Staff (20 points)
                                  bwilson Jun 17, 2015 9:34 AM (in response to dshirley)

                                  Thanks for the bug report about using GENERATE_MASTER_OBJECT_FILE. We'll take a look at that.

                                   

                                  When you use assembly files, you're basically opting out of using bitcode for just those files. Even though there is a general requirement that bitcode is an all-or-nothing thing, i.e., all the 3rd party libraries you use must include bitcode, we make a special exception for assembly code. You still need to build assembly files with clang -fembed-bitcode, though.

                                   

                                  Bitcode can be embedded in both static and dynamic libraries. In either case, you need to build them with an Archive action in Xcode (or from the command line, using "clang -fembed-bitcode" or "swiftc -embed-bitcode").

                                    • Re: Bitcode and Assembly?
                                      Danoli3 Level 1 Level 1 (0 points)
                                      Danoli3 Jul 9, 2015 8:20 AM (in response to bwilson)

                                      Thanks! -fembed-bitcode is exactly what I needed to build some static libraries.

                                        • Re: Bitcode and Assembly?
                                          Level 1 Level 1 (0 points)
                                          stack_tom Jul 11, 2015 12:01 PM (in response to Danoli3)

                                          How do you actually use the bitcode libraries for archive? I get the error:

                                           

                                          ld: warning: ignoring file XXXX/XXXX, file was built for archive which is not the architecture being linked (arm64).

                                            • Re: Bitcode and Assembly?
                                              Level 1 Level 1 (0 points)
                                              stack_tom Jul 11, 2015 11:44 PM (in response to stack_tom)

                                              I just built a library in Xcode and realized that it builds for architectures of armv7 and arm64 when you make a bitcode library. I'm assuming this is why mine isn't working, as I was only using an armv7 bitcode library. However, I thought the whole purpose of bitcode was to eliminate fat binaries/libraries. Can someone explain what I'm misinterpreting?

                                                • Re: Bitcode and Assembly?
                                                  bwilson Apple Staff Apple Staff (20 points)
                                                  bwilson Jul 12, 2015 2:54 PM (in response to stack_tom)

                                                  Yes, the armv7 and arm64 architectures are still separate, even when using bitcode. That would explain why it wasn't working for you. Bitcode does not eliminate the need for "fat" libraries, at least for the existing architectures.

                                  • Re: Bitcode and Assembly?
                                    Apfelmaennchen Level 1 Level 1 (0 points)
                                    Apfelmaennchen Oct 23, 2015 5:56 AM (in response to ChasingPtr)

                                    One aside:

                                     

                                    > 2. When using Xcode, bitcode is only produced during archive build, not debug or release build.

                                     

                                    The build setting that causes Xcode to pass `-fembed-bitcode` to the compiler in an archive build, but not in a release, or debug build, is `DEPLOYMENT_POSTPROCESSING`. Setting this to `YES` for your release configuration in combination with `ENABLE_BITCODE = YES` will cause Xcode to generate bitcode in regular release builds, as well.

                                    • Re: Bitcode and Assembly?
                                      Ben Hirschberg Level 1 Level 1 (0 points)
                                      Ben Hirschberg Nov 16, 2015 3:51 AM (in response to ChasingPtr)

                                      You wrote: "For iOS, this should *just work*."

                                       

                                      What about tvOS? Or Apple Watch?

                                       

                                      Many thanks!

                                    • Re: Bitcode and Assembly?
                                      Tia Lapis Level 2 Level 2 (75 points)
                                      Tia Lapis Jun 14, 2015 10:47 AM (in response to Colin Cornaby)

                                      "Bitcode seems to leave open the possibility that our application could end up running on a platform who's capabilities we did not optimize for in advance."

                                       

                                      That is exactly the reason for bitcode as I see it. Come OSX 10.12 with bitcode delivery, too and *peng* there is the infrastructure for ARM in Macs...

                                      • Re: Bitcode and Assembly?
                                        Level 1 Level 1 (0 points)
                                        ssb_SFNT Jun 16, 2015 12:35 AM (in response to Colin Cornaby)

                                        You potentially might even be more concerned about the fact, that bitcode is much easier to reverse then code lowered to a CPU (assembly).

                                        You may trust Apple, that submitted bitcode won't get inspected or may leak, but in case of preparing Frameworks for 3rd parties, I would not be that sure about this.

                                         

                                        Looks like Apple had this use case not in mind or is not considering reverse engineering on 3rd party framworks as a concern.