Post

Replies

Boosts

Views

Activity

Reply to Parallel processing app distributed by developer won't run sandboxed
I thought I would report progress and success, with some details in case anyone else is having similar troubles. I hope Quinn or someone else more knowledgeable than I will correct me if my report is misleading. I have successfully created and made available for non-App-Store distribution, a sandboxed version of my application, for which Gatekeeper observes that Apple has been able to check the app for malicious software and found none. Huzzah ... All the information about how to do that was available here or on other Apple sources, but there has been a lot of confusion about notarization, so it took some effort for a relative newcomer, like me, to put the pieces together, the more so since my app is a bit weird. First, it was clear from various postings (including some of Quinn's) that one must treat the app bundle as a tree rooted at "MyApp.app", and work from the leaves to the root, signing anything that needed signing. My app had several embedded binaries, so I did this by CDing to the directories that contained them, before building, and using codesign --force --options runtime --sign "Developer ID Application: <my name> <my team ID>" ./<embedded binary> I set up a script to do that at build time. I let Xcode manage signing the app, and enabled sandboxing with the entitlements I needed. So far so good. I was a blindsided for a while by having had to do the extra signing as a separate task: I did not realize that the "Distribute App" button at the top right corner of the Xcode (14.3.1) Organizer window would walk me through the notarization process, notwithstanding that I had had to pre-sign several embedded binaries beforehand. But once I had made an archive of the app (also an Xcode task), that button did its job, and the process provided useful messages when I had not gotten everything straight, so that notarizing actually turned out to be pretty easy once I got used to it. I also notarized a couple of disc images for distribution -- one with just the app and one with source code. To do so, I compressed the disc images via the "compress" option in the menu that pops up when you control-click on something in Finder. To notarize, I used "notarytool" from the command line, and the command that worked for me was (deep breath): xcrun notarytool submit <path to compressed .dmg> --wait --apple-id <my id> --password <specialized app password> --team-id <team-id number> Getting those command arguments just right was a bit of a pain -- your mileage may vary. That done, I stapled the ticket to the disc image (not the compressed version), also from the command line, via: xcrun stapler staple <path to disc image> I hope all this makes sense. I will have to experiment with NSWorkSpace to see if I can also enable hardened runtime. That will take some time, and it may turn out that hardened runtime will not work at all. I may report here further, depending on what I learn.
Aug ’23
Reply to Parallel processing app distributed by developer won't run sandboxed
Quinn, thank you for your thoughtful reply. I will explore more secure options for hardened runtime, will report here in case of difficulty, and will keep my fingers crossed about future requirements for distributing un-notarized apps outside the Mac App Store.. However ... For the long term, I am thinking that it might be fun if I could distribute my app through the Mac App Store. I expect that would be an uphill battle, but one obvious first step is getting it to run when sandboxed. So if you have any hints or suggestions for how to do that, or for helpful documents I might usefully read, please let me know. On the theory that a picture is worth a thousand words, I include (if the system will allow it) a link to my own web site which shows (scroll down) some screen shots of the app running, and which has other links to lots of documentation about it: http://www.jayreynoldsfreeman.com/My/Wraith_Scheme_%2864-bit_version%29.html You state: ... it’s clear that you’re doing some wacky stuff ... Nice to know that we mere out-in-the-field developers can still surprise the folks at Apple. :-)
Jul ’23
Reply to Memory mapped file: "Cannot allocate memory"
I stumbled across this thread rather by accident, but it reminded me of a problem I have been working lately, so I thought I would describe my situation in case the details were of interest. Specifically, I seem to have run across a 2.5 GByte limit in creating mmapped files under macOS (not iOS), and I have some details about what is going on. I was working on a macOS app using an Intel Mac Pro as development platform, running Big Sur, using the appropriate Xcode. I intended to develop an app that would run on both Macs with Intel processors and Macs with "Apple silicon" processors, but to get going I was only worrying about the Intel side of things. I wanted to mmap a large area at a virtual address that I needed to specify at compile time (I will spare you the details of why, unless someone is interested). My approach was to use .zerofill directives in a .s file that was part of the build, like this: .zerofill __WSS, __Wss000, _WraithSpace, 0x7f000000LL and then add entries under "Other Linker Flags" in the Xcode project settings, like this: -segaddr __WSS 0x20000000000 That combination creates a section named __Wss000 in the BSS of my executable, of size 0x7f000000, at virtual address 0x20000000000, which is just what I intended. I confirmed its existence by invoking "objdump -h" on the Intel version of my executable (more about the Apple silicon version later) -- which is way down in the Contents/MacOS directory inside my app's package. "objdump -h" prints out (among other things): Sections: Idx Name Size VMA Type [...] 30 __Wss000 7f000000 0000020000000000 BSS I could then mmap using the known address and size. I didn't pick the section size of hex 7f000000 by choice: I wanted a larger section, but when I used larger sizes, the Xcode build refused to put them in the executable. The output of objdump -h was vastly different -- there was a section called "HUGE" that I did not know what to do with, the section I wanted was not there, and the app didn't work. I played around with the size, noticed that the largest size that would work was something near 2.5 GByte (7f000000 hex is 2.13 GByte), and made do with that. I suspect that I had run up against a version of the 2.5 GByte limitation mentioned at the start of this thread, but ... NOTE TWO THINGS: (1) This was developing an app for macOS, not iOS. (2) The 2.5 GByte limit apparently did not exist on Apple silicon! I ended up using the "TargetConditionals.h" header (which Apple provides) and some preprocessor directives in my .s file, like this (actual code from my source files): #import "TargetConditionals.h" #if TARGET_CPU_X86_64 .zerofill ClowderSegment, ClowderSector, _ClowderSpace, 0x00f00000LL .zerofill __WSS, __Wss000, _WraithSpace, 0x7f000000LL #elif TARGET_CPU_ARM64 .zerofill ClowderSegment, ClowderSector, _ClowderSpace, 0x2000000000LL .zerofill __WSS, __Wss000, _WraithSpace, 0x2800000000LL #endif This code created the small segment in the Intel executable, that I just described, but when I moved the app to the M1 Mac Mini I had bought for testing, and ran objdump -h on the executable there I found a segment 0x2800000000LL long (that's 171 GByte) at 0x2000000000, and my app can use it with no problems (there are of course some similar #ifdefs here and there in the C++ code to be sure that the two different executables each are using the correct addresses for mmapping and such.) Oh, and there's ONE THING MORE ... What I was actually doing was getting around to updating a rather old app to run on both kinds of processors. In fussing with the old code, I noticed that some version of the "2.5 GByte limit problem" appears to have been around for more than a decade. I have not investigated in detail and I do not remember all the details, but I have been having problems with mmapping large chunks of memory since the days of macOS 10.6 ("Snow Leopard")! In case any of you are terminally curious, the app is a parallel processing Scheme interpreter (Scheme is a version of Lisp), and the big mmapped chunk is Scheme main memory shared between all the versions of the interpreter that are running in parallel. Google "Wraith Scheme Jay Freeman" and you will probably find it.
Sep ’22
Reply to Xcode won't build for universal binary
I am the original poster; I have more information and have sort of solved the problem, but there may be a problem with Apple's documentation or with Xcode, so I comment further here. My problem was that the Xcode build setting "VALID_ARCHS" was set to just "x86_64". When I set it to "arm64 x86_64", I was able to build for both architectures, and my app ran on both kinds of silicon. However: (1) There was no mention of "VALID_ARCHS" in Apple's documentation at https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary. Between that lack, and the fact that I had forgotten that the "VALID_ARCHS" setting existed, I didn't think to look for it, and only stumbled across it after a lot of thrashing. (2) Various documentation appears to indicate that the "VALID_ARCHS" setting is going away. If so, should Xcode have paid attention to it?
Mar ’22
Reply to Embedded Assembler in Xcode C++ routines?
I am the original poster of this question. I have made some progress on this one on my own, and since the matter comes up now and then I thought it would be useful to post some useful links I have found, as an aid to others. What follows is a raw list of links, but I think it will be pretty obvious on opening them why I found them useful, and I shall have some further comments on my learning process after the list ... (And the list will probably suggest to you that the reason I wanted to use assembler in the first place had to do with interprocess synchronization.) ======== Start of list of links ======== Bibliography for parallel process synchronization: x86-TSO: A Rigorous and Usable Programmer’s Model for x86 Multiprocessors Peter Sewell Susmit Sarkar Scott Owens University of Cambridge University of Cambridge University of Cambridge Francesco Zappa Nardelli Magnus O. Myreen INRIA University of Cambridge        http://www.cl.cam.ac.uk/users/pes20/weakmemory https://developer.apple.com/documentation/driverkit/3131285-ossynchronizeio by googling "mfence macintosh" https://www.cl.cam.ac.uk/~pes20/weakmemory/cacm.pdf Intel's official stuff: https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html Embedding assembly in C: https://clang.llvm.org/compatibility.html https://llvm.org/docs/LangRef.html https://llvm.org/docs/CommandGuide/llvm-as.html https://en.wikipedia.org/wiki/GNU_Assembler https://cs.lmu.edu/~ray/notes/gasexamples/ https://www.cs.uaf.edu/courses/cs301/2014-fall/notes/inline-assembly/ https://gcc.gnu.org/onlinedocs/gcc-4.0.2/gcc/Extended-Asm.html#Extended-Asm https://en.wikipedia.org/wiki/X86_calling_conventions https://stackoverflow.com/tags/inline-assembly/info https://en.wikipedia.org/wiki/X86_calling_conventions https://www.agner.org/optimize/calling_conventions.pdf https://developer.apple.com/forums/thread/64494 ======== End of list of links ======== Further comments: I encountered several issues in my investigations, most of which had to do with the fact that the CLang compiler was smarter than I was. The first issue, however, was simply that I hadn't written any x86-family assembler since the time when the newest, hottest chip in the Intel family was a wonderful ***** with full 16-bit internal data paths, called the 8088: In connection with its friends the 8086 and the 8087, Intel was near the head of the pack in advanced microprocessors in the late 1970s. In particular, I did not know that there were different names for the registers on later-model Intel processors, corresponding to whether you wanted to access 8, 16, 32 or 64 bits of them. This led to a lot of syntax errors, which were certainly mostly due to my own ignorance, but I might mention that the error messages reported by the assembler weren't much more useful than the classic yacc "Syntax error" message. With that one out of the way, I then was confronted with trying to outsmart the compiler. I was using embedded assembler of the canonical form __asm__( "this" "that" "the other" : "=r" (anOutput) : "r" (anInput) : "%this-register-is-dead-in-the-water" ); all embedded in a little C++ function, using the assembler code to examine the function's inputs and change its outputs. That function was temporarily installed in the actual Xcode project I was developing, so I could be sure the set of options and flags that affected the assembler were the same as for the rest of the project. (I was, of course, planning to use assembler in that project, so that condition was appropriate.) The function was declared and defined in one place and called from somewhere else, early in my project's initialization code, so I could print out results to a log file and see what was happening. I had figured out the compiler's conventions for register use in calling C functions, but the correct registers did not seem to contain sensible values -- or indeed, any relevant values at all. Eventually, a little fussing with "otool -tvV" (see list items above) (and much easier than using the debugger) led to the discovery that the compiler was in-lining my little function: The calling conventions were irrelevant because there was no call in the first place. Fortunately, there is a directive to prevent in-lining of functions that seems to work. It looks like this, e.g.: void __attribute__ ((noinline)) foo( int whatever ) { /* ... regular C code and assembler herein ... */ } With that out of the way, I still had problems, and more use of otool -tvV revealed that evidently, the compiler was smart enough to figure out that there was only one call to my little function in the entire body of my big project, and although it did not inline the call, nevertheless it modified the function to contain some of the literal values that were used where that call appeared in the source code. Thus my function happened to be called with one parameter equal to "2", and instead of pushing "2" into a register for the call, the compiler simply inserted a literal "2" wherever it was required in the assembled code for the body of my function. At least, that is what it looked like. What a wonderful optimization, but not what I wanted for purposes of experiment. I might have tried dialing back on optimization (I was running -O3 because that is what I have been using in my project), but it was easier and more consistent with my intentions of using assembler, simply to arrange to call my little function twice, with different literal values. At that point, the compiler gave up and generated the kind of call-with-register-use that all the books tell you to expect. So for the moment I have outsmarted the compiler and am able to proceed with my project, though I suspect it is plotting other surprises for me. I may modify this comment later, depending on what I find out, but in the interim, those of you who wish to use in-line assembler in MacOS applications created with Xcode should be advised that the matter is subtler than you think, even if you already think it is pretty subtle.
Dec ’21
Reply to Nib file not showing up in Xcode 13.0 interface builder
The storyboard is not scrolled off view, and what is very strange is, that when I pull down the Editor menu with the nib view selected, it does not have a Minimap entry -- it is not that the entry is grayed out, there is no such entry. I have not tried to create a new project, but the problem seems to occur with several different existing nib files. I have not seen this problem in Xcodes previous to Xcode 13. I find no relevant errors in the issues navigator. In particular, I ran a full analyze and no messages came up concerning the nib file. And as I said before, the builds complete with no errors and the product runs as expected. I have MORE INFORMATION: When I do the following: 1) Close Xcode. 2) Locate the nib file in question in the finder. 3) Control-click the nib file in the finder, and select "Open with Xcode" ... Then Xcode opens the nib file in a different kind of window than the one that opens when I open a project. This "different" window shows just the nib file. This window perhaps resembles Interface Builder windows from the days when the Interface Builder was a separate application. The window shows the nib file and allows me to edit it, and changes I make there show up when I (subsequently) open the project, then build and run it. But I still cannot view the nib file in the project viewer, even after I have edited it in the different window, and closed and reopened Xcode. I have noticed one odd thing. In the "different" window, the editor in that window identifies the nib by name and has a "Type" pull-down menu, which shows "Default -- Interface Builder Cocoa NIB". But when I look at the nib file in the project-viewer window, the editor there does not have a "Type" menu at all. I conjecture that Xcode's project-viewing system has somehow gotten the wrong idea of what the nib file's type is (since Xcode's build system evidently can deal with the file, and the "different" window can also deal with it). But I may be wrong, and have no idea how to debug further.
Oct ’21