Detecting library hooking

macOS does have a setting to disable debugging: https://developer.apple.com/library/archive/qa/qa1361/_index.html (and which itself can be circumvented) and does offer hardened run-time: https://developer.apple.com/documentation/security/hardened_runtime

If you're already using that and want "more"...

There are papers and write-ups on reverse-engineering and anti-reverse-engineeringposted around the 'net.

Some folks will use a licensing dongle, whether USB or NFC or otherwise.

Some vendors will use a support-based licensing scheme, and which requires periodic contact with servers you control.

If the crack is loading exploit code at run-time (and somehow the hardened run-time isn't an option or isn't working for your case), you could submit known-bad license keys from various spots in the code (after the first "real" check, and fail (and preferably failing well after the good and bad checks, and preferably failing with an obscure run-time error selected from a list of reserved-for-cracks error codes, and with a request for the user to contact your support organization) when a bad license check succeeds. Various ways to build on this, too.

You could have your legal folks have a discussion with the GitHub legal folks too, but the code will move elsewhere.

But if your servers are down when an online registration happens, or when your copy-protection mis-triggers, legitimate customers will be cranky.

Cracks will always exist, and expending time and effort thwarting those is, unfortunately, wasted effort. And effort that can cause paying customers to be prevented from accessing your app.

For some background on past cracks, look up the history of a pre-Y2K crack known as DeCSS.

Preventing debugging is trivial, hooking is a different *****. In hooking the symbol table for the dynamically linking methods/functions is altered.I want to be able to know when that table has been changed! Look at the code in https://github.com/GGossip/InjectLib, /macOS-InjectPluginCode/InlineInjectPlugin/InlineInjectPlugin.m

        //Adobe Illustrator
        if (checkAppVersion("27.5.0")) {
            NSLog(@"Loading com.adobe.illustrator 27.5.0");
            hookPtrA(0x100BF9F84, ret1);
        }
```BOOL hookPtrA(intptr_t addr, void *replaceMethod, void **retOriginalFunctionAddress) {
    return hookPtr(0, addr, replaceMethod, retOriginalFunctionAddress);
    }

int ret1(void) { NSLog(@"==== return value 1."); return 1; }

The code that is supposed to check for a valid license, by simply changing the pointer in the link symbol table the licensing is bypassed. This may not seem like a big issue BUT hooking dynamic libraries is very common and very dangerous. For example Pegasus/Trident used it to achieve persistence. I was on the team at LookOut that did the initial reverse engineering on Pegasus/Trident.

What I'm looking for is a way , at runtime, to verify the symbol table hasn't been altered. So there are no "bad licenses" to check for! We are using a hardened runtime.

One thing I am recommending is that the license checker doesn't return a bool, I was thinking something like a key modded by something variable, but that we can verify.