I'm using some CoreML models from C++. I've been trying to profile them using the CoreML Instrument in Instruments. It seems that that only works when I sign my binaries with the get-task-allow entitlement.
Is there an easier way? Ideally I'd like to be able to profile a Python program that calls my C++ code and I would rather not re-sign Python.
Ideally I'd like to be able to profile a Python program that calls my C++ code
How did you install that Python interpreter?
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
I'm using python3.10 installed via homebrew. I just tried with /usr/bin/python3—I believe that's the one that comes with Xcode—but it also doesn't have any entitlements nor show data for the CoreML instrument.
I'm using python3.10 installed via homebrew.
OK, that’s actually a good thing. Doing this sort of stuff with the built-in Python (well, the one that comes is Xcode) is harder.
If you run the following command against your copy of Python, what do you see:
% codesign -d -vvv --entitlements - /path/to/your/python
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
I get this:
❯ codesign -d -vvv --entitlements - /path/to/my/python
Executable=/opt/homebrew/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/bin/python3.10
Identifier=python3-555549447295e55955d43aca9820d95d1f4ae151
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=489 flags=0x2(adhoc) hashes=9+2 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha256=f8edaad9c426fe4e51e0f52ffd1e2b2f7fa6126d
CandidateCDHashFull sha256=f8edaad9c426fe4e51e0f52ffd1e2b2f7fa6126d980124e04a63adbbac49d34e
Hash choices=sha256
CMSDigest=f8edaad9c426fe4e51e0f52ffd1e2b2f7fa6126d980124e04a63adbbac49d34e
CMSDigestType=2
Launch Constraints:
None
CDHash=f8edaad9c426fe4e51e0f52ffd1e2b2f7fa6126d
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements count=0 size=12
Yeah, that’s what I was expecting. If you run LLDB against that, can you execute it?
% lldb /path/to/python
(lldb) target create "/path/to/python"
Current executable set to '/path/to/python' ….
(lldb) r
… what happens here? …
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
Not entirely sure what should happen, but I get this:
❯ lldb env/bin/python
(lldb) target create "env/bin/python"
Current executable set to '/path/to/env/bin/python' (arm64).
(lldb) r
Process 33134 launched: '/path/to/env/bin/python' (arm64)
Process 33134 stopped
* thread #2, stop reason = exec
frame #0: 0x0000000100014a40 dyld`_dyld_start
dyld`:
-> 0x100014a40 <+0>: mov x0, sp
0x100014a44 <+4>: and sp, x0, #0xfffffffffffffff0
0x100014a48 <+8>: mov x29, #0x0
0x100014a4c <+12>: mov x30, #0x0
Target 0: (Python) stopped.
(lldb)
Separately, I tried signing my python binary like this:
❯ cat entitlements.xml
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.get-task-allow</key>
<true/>
</dict>
</plist>
❯ codesign -f -s - env/bin/python --entitlements entitlements.xml
env/bin/python: replacing existing signature
❯ codesign -d -vv env/bin/python --entitlements -
Executable=/opt/homebrew/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/bin/python3.10
Identifier=python3-555549447295e55955d43aca9820d95d1f4ae151
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=649 flags=0x2(adhoc) hashes=9+7 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements count=0 size=12
[Dict]
[Key] com.apple.security.get-task-allow
[Value]
[Bool] true
When I signed my C++ program this way, the CoreML instrument did work. However after signing python like this, it still does not work.
Also, signing python this way doesn't change the lldb output I get the same results as before signing for both lldb env/bin/python and lldb /opt/homebrew/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/bin/python3.10.
Not entirely sure what should happen, but I get this:
I was hoping that it’d actually run Python, but it does not. Rather, it’s stopping in some non-obvious way.
Notably, it actually works if you run bin/python from Terminal.
I ran some more tests in my office and worked out what’s going on. bin/python is not actually Python. Rather, it’s a trampoline that bounces over to Resources/Python.app/Contents/MacOS/Python. When you re-sign bin/python, it only resigns that tool. However, that tool simply execs Resources/Python.app/Contents/MacOS/Python. What happens there depends on your exact setup:
In my case Python.app had incompatible code signing, so things failed and I got a crash report.
In your case I think you’re just stopping after the exec.
Either way, it suggests a path forward. Rather than re-signing bin/python, re-sign Resources/Python.app. For example:
% cat tmp.entitlements
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.get-task-allow</key>
<true/>
</dict>
</plist>
% codesign -s - -f --entitlements tmp.entitlements Python.app
Python.app: replacing existing signature
And you can now run Python under the debugger:
% lldb Python.app/Contents/MacOS/Python
(lldb) target create "Python.app/Contents/MacOS/Python"
Current executable set to '/Users/quinn/Python3.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python' (arm64).
(lldb) r
Process 19846 launched: '/Users/quinn/Python3.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python' (arm64)
Python 3.8.9 (default, Apr 13 2022, 08:48:06)
[Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
I’m hoping that’s enough to make Instruments happy, but I wasn’t able to test that far.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
Ahh that's interesting. I tried with my homebrew binary:
codesign -f -s - /opt/homebrew/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/Resources/Python.app/Contents/MacOS/Python --entitlements entitlements.xml
I get the same results in lldb. I was also able to test the CoreML Instrument and it does work after re-signing like this. Thank you!