Notarized python app crashes with MemoryError

In my experience, there are two ways to approach problems like this:

• Apply random entitlements in the hope that you eventually stumble across one that works (A).

• Dig into the code to see which specific API failed, and what error it returned (B).

I generally recommend B over A.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Unfortunately B implies trying to understand why a library function like reset_cache is failing . Is there any way I can figure out which entitlement needs to be there. If it helps, the above entitlement worked once after which it has been failing

/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/__init__.py

Sorry, but you are a long way from a simple entitlement problem. Theoretically speaking, it possible to construct, sign, and notarize an app written in Python. But this is a monumentally difficult task. It is much more difficult than simply rewriting it in Swift.

For those reading along at home, I’m going to be helping zahsalim in a different context.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

(s. 730317854)

My point is that there is some inconsistency in the way the entitlements work. When I enabled the entitlement for the very first time, the app seemed to run without any issues. However, after that I have not been successful. I managed to go ahead by commenting out the line in ctypes since that seemed to be something which was needed only on windows platforms. However this resulted in another crash with MemoryError in the ffi library. The library maintainers ask us to use the com.apple.security.cs.allow-unsigned-executable-memory entitlement. Inspite of that the app continues to crash. Any advice other than rewriting the whole app in swift which is not feasible given timelines.

What exactly is your timeline then? Because your trace back lists files in two different locations, both of which are wrong for a bundled version of Python. You are having issues because your app is running Python code from all over your system. The only way you will be able to test this is on a pristine system like a VM, with absolutely no 3rd party Python additions installed.

You will need to build your own Python and all supporting modules, installed into your own prefix. Then you have to bootstrap your app and carefully construct your environment so that it sees only your embedded Python. Even a single hard-coded path, in any part or module, breaks it. Even doing this inside a default, 3rd party install of Python, without embedding it into an app, is a serious challenge.

I suggest you put aside the idea of Notarization for a while. You have to figure out how to get your app running on a different machine. You don't need notarization for that. Then, work on signing it. Don't try to hack the signing with --deep or --force. If you need those, then you have constructed your bundle incorrectly. Rebuild and retest. I'm not sure how far along this path you might be or your level of expertise with Python. Regardless, you will be a true expert in Python when you're done. Good luck!

Did anyone have any further luck dealing with this error?

I'm also seeing the same MemoryError issue also with _reset_cache on line 273 of ctypes/__init__.py, also on a notarized app but frozen with PyInstaller.

However, for me the code works on two test machines (OSX 10.13 and 10.15) and fails on two user machines (both 10.15). I have utterly no idea what might be the difference between them. One test machine, on which the software runs without fault, is a clean install hosted by MacinCloud.

For those who read this thread in the future...

The real problem I had was potentially very similar to the OP's real problem: both of us had issues with pollution of hard-coded links into the built package. Check out the 6th line, rather than the final line, in the error message I was seeing:

Code Block 
Traceback (most recent call last):
  File "PyInstaller/loader/pyiboot01_bootstrap.py", line 129, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/Users/[username]/.local/share/virtualenvs/[projectname]-ZWaEGgic/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 623, in exec_module
  File "ctypes/init.py", line 551, in <module>
  File "ctypes/init.py", line 273, in _reset_cache
MemoryError
[36334] Failed to execute script pyiboot01_bootstrap

I have no idea why the test system was consistently able to successfully run the app. However the users' systems would consistently produce the same error message.

Potentially the error was caused because the project was installed in editable mode (i.e. pipenv install -e .). However uninstalling the project and reinstalling it in the standard manner did not fix the issue.

The solution was to create an entirely new virtual environment, with the packages installed in the standard (i.e. not editable) manner. Within that environment I ran PyInstaller, codesign and notarized the app. The final package ran on all four machines.

I hope that helps the next person.

Matthew