The OpenSSL library interface to Allegro Common Lisp system stopped working with macOS 15.x (15.0.1 and 15.1).
We have tried many versions of OpenSSL. 1.1.1t (which we built ourselves), 3.0.x, 3.3.x, 3.4.0. All work fine on macOS 14 and earlier. All fail on macOS 15.
What is bizarre about the failure: we can load the SSL libraries fine, but when we try to make an outgoing connection it fails (with varying errors). Also, trying to use lldb to debug just hangs, once we step into the SSL libraries.
More specifically, using Homebrew OpenSSL 3.0.15 gives an exception that we see in lldb, but we cannot step into SSL_ctrl(), which is in libssl.3.dylib, provided by Homebrew.
We have also tried a version of OpenSSL 1.1.1t that we built ourselves (and codesigned and is included in the notarized app), and it fails with a SEGV, rather than the error below, which is using 3.0.15:
What started this were errors using the OpenSSL libraries. Here's the use case:
cl-user(2): (net.aserve.client:do-http-request "https://franz.com")
(net.aserve.client:do-http-request "https://franz.com")
Error: Received signal number 0
[condition type: synchronous-operating-system-signal]
Restart actions (select using :continue):
0: Return to Top Level (an "abort" restart).
1: Abort entirely from this (lisp) process.
[1] cl-user(3): :zo :all t :count 5
:zo :all t :count 5
Evaluation stack:
... 5 more newer frames ...
(excl::SSL_ctrl 6133462816 55 ...)
(excl::ssl-device-open-common #<excl::ssl-client-stream closed fd # @ #x3079fed32> nil ...)
->((method device-open (excl::ssl-client-stream t t)) #<excl::ssl-client-stream closed fd # @ #x3079fed32> t ...)
((:internal (:effective-method 3 nil nil nil t) 0) #<excl::ssl-client-stream closed fd # @ #x3079fed32> t ...)
((:runsys sys::lisp_apply))
[... excl::function_lisp_apply ]
(excl::caching-miss #<standard-generic-function device-open> (# t #) ...)
[... device-open ]
... more older frames ...
[1] cl-user(4):
If you want to see the problem for yourself, I created a new, signed and notarized version of our application https://franz.com/ftp/pri/layer/acl11.0express-macos-arm64.dmg.
To use it, install Homebrew and do brew install openssl@3.0, then execute the following to get the error:
cd /Applications/AllegroCL64express.app/Contents/Resources
env ACL_OPENSSL_VERSION=30 DYLD_LIBRARY_PATH="$(brew --prefix openssl@3.0)/lib:$DYLD_LIBRARY_PATH" ./alisp
(progn (require :ssl)(require :aserve))
(net.aserve.client:do-http-request "https://franz.com")
You should get the error shown above.
Here's what we see when we set a breakpoint at SSL_ctrl:
lldb alisp
_regexp-env ACL_OPENSSL_VERSION=30
_regexp-env DYLD_LIBRARY_PATH=/opt/homebrew/opt/openssl@3.0/lib:
br s -n SSL_ctrl
run
(progn (require :ssl)(require :aserve))
(net.aserve.client:do-http-request "https://franz.com")
Then, we see this:
cl-user(2): (net.aserve.client:do-http-request "https://franz.com")
(net.aserve.client:do-http-request "https://franz.com")
Process 5886 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.2
frame #0: 0x0000000102081090 libssl.3.dylib`SSL_ctrl
libssl.3.dylib`SSL_ctrl:
-> 0x102081090 <+0>: stp x20, x19, [sp, #-0x20]!
0x102081094 <+4>: stp x29, x30, [sp, #0x10]
0x102081098 <+8>: add x29, sp, #0x10
0x10208109c <+12>: mov x20, x2
(lldb) si
<<<hang here>>>
Again, it only started with macOS 15. We have not seen this on any previous version.
More detail:
$ codesign -vvvv /Applications/AllegroCL64express.app
/Applications/AllegroCL64express.app: valid on disk
/Applications/AllegroCL64express.app: satisfies its Designated Requirement
$
$ codesign -d --entitlements - /Applications/AllegroCL64express.app
Executable=/Applications/AllegroCL64express.app/Contents/MacOS/AllegroCL64express
[Dict]
[Key] com.apple.security.cs.allow-dyld-environment-variables
[Value]
[Bool] true
[Key] com.apple.security.cs.allow-jit
[Value]
[Bool] true
[Key] com.apple.security.cs.disable-library-validation
[Value]
[Bool] true
[Key] com.apple.security.get-task-allow
[Value]
[Bool] true
$
The other thing we noticed in debugging this is even though we set DYLD_LIBRARY_PATH, another libssl seemed to be found by lldb. For example, in this case 3 versions of SSL_new were found by lldb:
$ lldb alisp
(lldb) target create "alisp"
Current executable set to '/Applications/AllegroCL64express.app/Contents/Resources/alisp' (arm64).
(lldb) _regexp-env ACL_OPENSSL_VERSION=30
(lldb) _regexp-env DYLD_LIBRARY_PATH=/opt/homebrew/opt/openssl@3.0/lib:
(lldb) br s -n SSL_new
br s -n SSL_new
Breakpoint 1: 2 locations.
(lldb) run
Process 6339 launched: '/Applications/AllegroCL64express.app/Contents/Resources/alisp' (arm64)
Copyright (C) 1985-2023, Franz Inc., Lafayette, CA, USA. All Rights Reserved.
...
CL-USER(1): (progn (require :ssl)(require :aserve))
; Fast loading
; /Applications/AllegroCL64express.app/Contents/Resources/code/SSL.002
...
T
CL-USER(2): (net.aserve.client:do-http-request "https://franz.com")
Process 6339 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.3
frame #0: 0x00000001020803ec libssl.3.dylib`SSL_new
libssl.3.dylib`SSL_new:
-> 0x1020803ec <+0>: stp x20, x19, [sp, #-0x20]!
0x1020803f0 <+4>: stp x29, x30, [sp, #0x10]
0x1020803f4 <+8>: add x29, sp, #0x10
0x1020803f8 <+12>: cbz x0, 0x102080700 ; <+788>
(lldb) br list
Current breakpoints:
1: name = 'SSL_new', locations = 3, resolved = 3, hit count = 1
1.1: where = libboringssl.dylib`SSL_new, address = 0x0000000193f1b160, resolved, hit count = 0
1.2: where = libssl.48.dylib`SSL_new, address = 0x000000026907f64c, resolved, hit count = 0
1.3: where = libssl.3.dylib`SSL_new, address = 0x00000001020803ec, resolved, hit count = 1
(lldb)
We are out of ideas on how to debug this.
Post
Replies
Boosts
Views
Activity
We have a native ARM64 application. The application is a development environment and native compiler for the language Common Lisp. CL has a foreign function interface, which allows loading of .dylib files into CL and calling functions in them from CL. For this reason, we add certain entitlements. See below.
It is notarized and installed on macOS 14.7. When I run spctl on it I get this:
$ spctl --assess -v /Applications/AllegroCL64.app
/Applications/AllegroCL64.app: rejected (the code is valid but does not seem to be an app)
That’s before I run it. Which is odd because the app is notarized. When I run the app, it asks for a license file and installs it into /Applications/AllegroCL64.app/Contents/Resources/ and after that, the spctl shows this:
$ spctl --assess -v /Applications/AllegroCL64.app
/Applications/AllegroCL64.app: a sealed resource is missing or invalid
I assume the mere act of copying the license (a file called devel.lic which is a small text file) is causing this. Why does it say it “does not seem to be an app”?
This self-modification of the files in the Contents/Resources directory is a huge feature. We allow downloading of patches, which add features and fix bugs in the product. Is this going to be a problem, going forward? I don’t remember seeing this result from spctl before and I have a feeling it’s a new , due to tightening of security policies, etc.
All of this is quite worrying to us.
More details of the app:
$ codesign -vvvv mlisp
mlisp: valid on disk
mlisp: satisfies its Designated Requirement
$ codesign -d --entitlements - /Applications/AllegroCL64.app
Executable=/Applications/AllegroCL64.app/Contents/MacOS/AllegroCL64
[Dict]
[Key] com.apple.security.cs.allow-dyld-environment-variables
[Value]
[Bool] true
[Key] com.apple.security.cs.allow-jit
[Value]
[Bool] true
[Key] com.apple.security.cs.disable-library-validation
[Value]
[Bool] true
[Key] com.apple.security.get-task-allow
[Value]
[Bool] true
$
Other details:
The app was built with the Command Line tools version 2395 on macOS 12.x.
This might seem flippant, but it's very serious. My use case, this is literally torturing me to death:
We have a language (Allegro Common Lisp) which has a module for making SSL connections. This module is requires OpenSSL. We don't want to supply OpenSSL with our product for obvious reasons--I'm not going to go into that here.
So, what's the problem? Well, first, the problem is that Apple decided long ago to not allow developers to build against their version of SSL (which is based on LibreSSL). So, we must depend on Macports or Homebrew for the installation of OpenSSL. Those libraries are in non-standard places and require DYLD_LIBRARY_PATH to be set so that the libraries can be dynamically loaded.
The problem is, with SIP, any environment variables that start with DYLD_ or LD_ are stripped from the environment passed to programs. That is, if I set DYLD_LIBRARY_PATH in BASH and start my product, I can't load a signed .dylib that depends on some version of the OpenSSL libraries.
To make matters worse, this works on macOS 11.6.5 (20G527) x86_64:
DYLD_LIBRARY_PATH=... ./mlisp
(sys:getenv "DYLD_LIBRARY_PATH")`
That is, it returns the value ... set on the command line.
It does not work on macOS 11.6.5 (20G527) arm64.
To dynamically load the OpenSSL libraries, we have a glue library (that adds functionality) called acliss11.dylib which depends on @rpath/libssl.1.1.dylib and @rpath/libcrypto.1.1.dylib.
For x86_64 we have a solution, but the same solution does NOT work for arm64.
How are we supposed to do this?