Right now it is used in a single threaded program. But the program comes as a shared library and is reentrant so it is possible multiple programs could be using the same shared library.

I think what you are telling me is there is no standard way other than setting signals to trap a memory access error. What I am doing, and it works, is using a signal handler to have a memory access error in my low level assembler memory access functions act like a regular error in the return code method of error reporting. What this means is, the handler traps the memory access error and forces the subroutine to exit with a return code indicating a memory access error occurred. What would be nice is if I could give the operating system a list of addresses where a bad memory access could occur, and a list of corresponding addresses of where I want execution to continue at link time. The problem is setting the signal handler and removing it before and after the memory accesses is slow. As for why I need to do this, in addition to just being good programming practice, I mean why not have your block move subroutine return an error code instead of just crashing? But the main reason is, my application is a Forth based script engine and the user can put in any addresses they want and it's very frustrating for me to try to debug my programs when they crash and I don't know why. An error message like, 'bad memory access at address 0x12823' along with a trace of all the nested functions that were in play is much better, which is what I have now. In short: What I am doing is I am making bad memory exceptions fit into the return code model of error handling using a signal handler. But, it's slow. Setting and unsetting the handler around each memory access address is time consuming. The option of setting the handler once during the initialization will not work because calling operating system functions and third party libraries can override my setting of the handler. But I think you've answered my question. There is no standard fast and easy way in Mac Os to trap memory exceptions in the manner I'm describing.

No, that’s not what I said. On Apple platforms there are two standard ways of handling memory access exceptions, UNIX signals and Mach thread/task/host exception handling. As to which of those is faster, I’ve never profiled them. I didn't know about the Mach thread/task/host exception handling. It looks like it will work for what I want to do. Thanks! IMO that’s terribly programming practice. What is the caller supposed to do with that error? If the program is in such a state that they’re passing bogus values to memcpy, the best thing to do is stop immediately. Any else is just asking for security exploits. It's not for the end user, it's for the programmer. This is to support the Forth playground style of debugging which allows programmers to make mistakes. For me I've found that staying in the playground is preferable to having my script engine exit. IMO that’s terribly programming practice. What is the caller supposed to do with that error? If the program is in such a state that they’re passing bogus values to memcpy, the best thing to do is stop immediately. Any else is just asking for security exploits. It's for the programmer. To help with debugging. I've found it really convenient. For example, let's say I forgot to put the script engine into the correct number base, or even more likely, I forgot to drop a return value off of the forth stack after calling a function. What happens is I get a bad memory error at some weird address, along with exactly which function I was in. Then I look at the stack and see the wrong numbers there... and go oh I forgot to do a DROP. If the program just crashes, it takes a lot longer to figure out and fix what was going on. I would solve this by using a suballocator for Forth memory, at which point you can reject bad addresses at the UI level, before they enter your runtime. I'm not sure I'm understanding your suggestion here. In my Forth the programmer can do something like: VARIABLE moo 0 0 moo D! The mistake here is, moo is a 64 bit variable and D! is a 128 bit operation. The programmer has just gone off the end of the buffer. Depending on how close they are to the end of the memory allocated by the operating system, it is possible to get a bad memory exception. I think it would be very complicated and expensive execution time wise to scan the address passed to D! against all the buffers allocated in my system. On the other hand, I did include a managed buffer system which uses buffer ids and offsets. All operations that use that system do a lot of checks to make sure no bad offsets or buffer ids get used, so bad memory errors are not possible with them. In any case, I will mark this as solved. The thing I found online about the Mach thread/task/host exception handling looks promising but was from 2013. They said there was an issue with documentation not being available for IOS... hopefully it's been resolved since then.

The key question here is whether you let your Forth engine interact directly with OS-level objects. Yes. That is kind of the whole point. If you do then the only safe option here is to run the developer’s code in a separate process. Remember that memory access exceptions aren’t the only potential outcome for rogue pointer accesses. If the developer’s code accidentally smashes the the memory being used by your runtime, things are going to end badly. Yes that can happen. I believe the shared library's code is read only after it is loaded. The engine's data memory can get corrupted. The low level buffer access routines in my engine's system checks to see if its management system has been corrupted and returns errors if it happens. In the time I've been working on my engine, which is more than 20 years, it's only happened a couple times. The ones that I remember were due to buffers overruns on a block moves that didn't generate bad memory exceptions and ended up going a little bit into the next buffer. I don't think running the engine in a separate process is going to work. Forth is an interactive extensible stack based programming language with little to no type checking. Development is done by extending the engine and requires access to the engine's data. Again, if the developer uses my managed buffer system through the access routines, bad memory accesses are impossible. But, the Forth Standard requires allowing direct access. And direct access is faster. Forth is pretty simple and there aren't that many system variables, and I could put those variables in a separate buffer... but... Forth is backwards compatible with 8 and 16 bit systems where there aren't multiple buffers, and expects the engine to start up in a certain way which has everything in one buffer. And adding new variables goes to the same buffer as the existing variables by default... Actually they are coming out with a new standard... maybe that's changed.... be nice if it did. But this would mean the only way to access the engine's data would be through some sort of remote procedure call? Wouldn't that be kind of slow? Forth uses a data stack, and my implementation is already kind of slow because of all the safety checks I do, like checking for underflows, overflows, etc. So I would put the data stack buffer in a separate process and then use remote calls to push numbers to the data stack, and the developer would write their function which is supposed to use the data on the data stack... and their function would have to use remote calls to manipulate the data... But if their function is a memory access... in order to get the protection, the memory access would have to be done from the developer's process and not the engine's process... Hmm... I'm guessing sending messages to other processes is involved here... The main issue is that Forth lets developers do whatever... accesses to data are not through accessor functions. Instead addressed are pushed onto the data stack and the developer is allowed to do calculations on those addresses. Then the functions to read and write memory are called which uses the addresses on the data stack. Sometimes the developer will be accessing their own memory, and sometimes they will need to access the engine's memory. There's really no way to tell the difference between the two without completely rewriting the Forth language to include a type flag with the addresses on the data stack... (Someone is doing something like that by the way.. but he's using it to tell the difference between integers and floating point data.)

Thank you for the reply Quinn. I think I understand your answer. I am using XCode and the command line tools and linking against the .tbd stub libraries in the SDK. But to do it I had to specify the path to the stub files since the path to the stub files does not lie along the standard shared library search path automatically. What I'm worried about, and it's probably nothing to worry about, is that someone who uses the compiled program and does not have XCode installed, will have problems linking to the libraries. I'm guessing when ld builds an executable and you use -lc to tell ld to link against the c standard library, the path to the library is not built into the executable. Instead, will the run time linker then use the shared cache for the libraries? I can't test this theory easily since I only have one machine and it has XCode with the command line tools installed. Uninstalling and reinstalling XCode and tools to test it is doable, but I still wouldn't know if the uninstall process completely removed everything... so I figured I'd ask you instead.

What happens when you try to compile without specifying that path? What error message do you get? What does your compile command look like? Compiling this code #include <unistd.h> const unsigned char pmessagebuf[13] = "hello world\n"; int main (int argc, char* argv[]) { write(STDOUT_FILENO, (void*)pmessagebuf, 12);      return(0);    } using these commands: cc -c ./testcprogram.cpp -mmacosx-version-min=10.11 -mstackrealign ld -execute -macosxversionmin 10.11 -lSystem -o testcprogram testcprogram.o Does not work. You get the error that the system library can not be found. Instead you have to use this linker command: ld -execute -macosxversionmin 10.11 -lSystem -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -o testcprogram testcprogram.o Don't you think that if there was a problem, someone would have noticed it by now? It is a known issue in Big Sur and is in the release notes. I only found out about the work around because a lot of others on the internet already figured it out and posted about it. The path to the library is is built into the executable. You can inspect that with "otool -L". The fact that the operating system uses the shared cache is an implementation detail. If that's true, then you can't use ld to link .o files that have calls to the system library for production software anymore because the libraries will not be able to be found when xcode and the command line tools are not installed. Don't worry. About 50 million other people have thoroughly tested it by now. If you do have concerns about how your app would run in a "factory fresh" computer, you can install a virtual machine like Parallels and test it there. I strongly recommend that. Many developers install dynamic libraries from various "ports" or "brew" tools, from crazy cross-platform development tools, and are completely helpless when their apps don't work properly anywhere except their development machines. As you can see, I'm not using a crazy cross-platform development tool to compile this code. I'm only using the XCode command line tools cc and ld. And yes, there is a problem. I was hoping to hear Apple has say. Like if this will be fixed in the next release, or if support for ld is dropped, or if somehow using -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib with ld will work for production software. But from what you are saying, it looks like -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib is not a good solution.

You are using ld directly to link an executable? Why? I've never seen anyone do that. It's C 101. OK. So? You are doing something totally unique and you found a workaround - for today at least. I know I'm getting old, but hopefully they still teach this in school. We're talking about using ld at its most fundamental level. ld is used to link things together, including the system library. What you are technically saying is Apple is dropping support for using ld to link production executables. I hope you don't work for Apple. I haven't see anything in the release notes about using ld. Maybe you are referring to the dynamic library cache issue. That is only incidentally related to ld.  Yes, I am referring to the dynamic library cache issue. This is what broke ld. The Apple work around for the dynamic library cache issue is to use dlopen. Unfortunately, if your executable has to link against the system library to gain access to dlopen, and an ld compiled executable can't 'see' the system library because it's in the cache, then you can't use ld to link production software anymore. If this is the case, then Apple should say it is dropping support for the command line tools and all code must be compiled using the XCode IDE. However, the -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib may work... I just have to test it. It's just ridiculous to have to tell ld what the path is to the libraries. The default paths for the ld linker tool should include everything it needs for linking against the system library once you install the command line tools. What I'm really curious is how the XCode IDE is linking production executables. They must not be using ld.

If you want to explicitly run the linker, you’ll need to tell it what SDK to use. A good way to work out the exact syntax is to run the compiler driver with -v: Code Block "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -ltolibrary /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch x8664 -platformversion macos 11.0.0 11.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o test -L/usr/local/lib test.o -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/lib/darwin/libclangrt.osx.a The ‘secret sauce’ here is -syslibroot. You can inspect that with "otool -L". Thanks Quinn and etresoft. I was able to link the clang generated helloworld.o file using this command: ld -demangle -dynamic -arch x8664 -platformversion macos 11.0.0 11.1 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o testcprogram -L/usr/local/lib testcprogram.o -lSystem I also verified with otool that the only library path for dyld and the system library in the executable is /usr/lib I was also able to link a helloworld .o file generated using an assembler I wrote. I noticed that -execute is no longer required for ld, and that -platform_version has replaced -mmacosx-version-min. I also couldn't find any documentation on -syslibroot. I'm guessing that the documentation for these changes will come sometime in the future.

I was also able to get this assembly language hello world to work: // Hello World for Big Sur .text _helloworldmsg: .asciz "Hello World!\n" .globl _myhelloworldstart _myhelloworldstart:     // syscall is deprecated.     // Last I checked, if you use syscall, the assembler compiles an imported function call.     //  If you hard code the syscall opcode, macos will not execute it and will give you an error     //  so I am using imported function calls to access the system library functions.     andq $0xfffffffffffffff0, %rsp // Force alignment. Without this exit will seg fault.     movq $1, %rdi     leaq _helloworldmsg(%rip), %rsi     movq $13, %rdx     call _write      xorq %rdi, %rdi // clear rdi to return success     call _exit using these commands: as -mmacosx-version-min=10.11 ./helloworldbigsur.s -o helloworldbigsur.o ld -e myhelloworldstart -nouuid -noehlabels -demangle -dynamic -arch x8664 -platformversion macos 11.0.0 11.1 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o helloworldbigsur -L/usr/local/lib helloworldbigsur.o -lSystem

Big Sur has moved the system libraries to a cache that is no longer in the file system. Eskimo from Apple answered how to figure it out for my 'how to link to the libraries in C' question in another thread, and that answer should work for a Nasm generated .o file as well. I also figured out how to do it using the gnu assembler that comes with XCode and this is how I did it: Using this gnu assembler assembly language source code: .text _helloworldmsg:.asciz "Hello World!\n" .globl _myhelloworldstart _myhelloworldstart:    &#9;// syscall is deprecated.    &#9;// Last I checked, if you use syscall, the assembler compiles an imported function call.    &#9;//  If you hard code the syscall opcode, macos will not execute it and will give you an error    &#9;//  so I am using imported function calls to access the system library functions.     andq $0xfffffffffffffff0, %rsp&#9;// Force alignment. Without this exit will seg fault.  // In other words, the system functions require that the return stack is 16 byte aligned before you call them.   movq $1, %rdi    leaq _helloworldmsg(%rip), %rsi    movq $13, %rdx    call _write     xorq %rdi, %rdi // clear rdi to return success    call _exit I was able to compile and link using these commands: as -mmacosx-version-min=10.11 ./helloworldbigsur.s -o helloworldbigsur.o ld -e myhelloworldstart -nouuid -noehlabels -demangle -dynamic -arch x8664 -platformversion macos 11.0.0 11.1 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o helloworldbigsur -L/usr/local/lib helloworldbigsur.o -lSystem It looks like /usr/local/lib is the path to the dynamic library cache even though the files are not in the file system. When I used otool to look at the executable, it looks like the system loads dyld from /usr/lib and then adds /usr/local/lib to the library search path.