The OpenSSL library interface to Allegro Common Lisp system stopped working with macOS 15.x (15.0.1 and 15.1).
We have tried many versions of OpenSSL. 1.1.1t (which we built ourselves), 3.0.x, 3.3.x, 3.4.0. All work fine on macOS 14 and earlier. All fail on macOS 15.
What is bizarre about the failure: we can load the SSL libraries fine, but when we try to make an outgoing connection it fails (with varying errors). Also, trying to use lldb to debug just hangs, once we step into the SSL libraries.
More specifically, using Homebrew OpenSSL 3.0.15 gives an exception that we see in lldb, but we cannot step into SSL_ctrl(), which is in libssl.3.dylib, provided by Homebrew.
We have also tried a version of OpenSSL 1.1.1t that we built ourselves (and codesigned and is included in the notarized app), and it fails with a SEGV, rather than the error below, which is using 3.0.15:
What started this were errors using the OpenSSL libraries. Here's the use case:
cl-user(2): (net.aserve.client:do-http-request "https://franz.com")
(net.aserve.client:do-http-request "https://franz.com")
Error: Received signal number 0
[condition type: synchronous-operating-system-signal]
Restart actions (select using :continue):
0: Return to Top Level (an "abort" restart).
1: Abort entirely from this (lisp) process.
[1] cl-user(3): :zo :all t :count 5
:zo :all t :count 5
Evaluation stack:
... 5 more newer frames ...
(excl::SSL_ctrl 6133462816 55 ...)
(excl::ssl-device-open-common #<excl::ssl-client-stream closed fd # @ #x3079fed32> nil ...)
->((method device-open (excl::ssl-client-stream t t)) #<excl::ssl-client-stream closed fd # @ #x3079fed32> t ...)
((:internal (:effective-method 3 nil nil nil t) 0) #<excl::ssl-client-stream closed fd # @ #x3079fed32> t ...)
((:runsys sys::lisp_apply))
[... excl::function_lisp_apply ]
(excl::caching-miss #<standard-generic-function device-open> (# t #) ...)
[... device-open ]
... more older frames ...
[1] cl-user(4):
If you want to see the problem for yourself, I created a new, signed and notarized version of our application https://franz.com/ftp/pri/layer/acl11.0express-macos-arm64.dmg.
To use it, install Homebrew and do brew install openssl@3.0, then execute the following to get the error:
cd /Applications/AllegroCL64express.app/Contents/Resources
env ACL_OPENSSL_VERSION=30 DYLD_LIBRARY_PATH="$(brew --prefix openssl@3.0)/lib:$DYLD_LIBRARY_PATH" ./alisp
(progn (require :ssl)(require :aserve))
(net.aserve.client:do-http-request "https://franz.com")
You should get the error shown above.
Here's what we see when we set a breakpoint at SSL_ctrl:
lldb alisp
_regexp-env ACL_OPENSSL_VERSION=30
_regexp-env DYLD_LIBRARY_PATH=/opt/homebrew/opt/openssl@3.0/lib:
br s -n SSL_ctrl
run
(progn (require :ssl)(require :aserve))
(net.aserve.client:do-http-request "https://franz.com")
Then, we see this:
cl-user(2): (net.aserve.client:do-http-request "https://franz.com")
(net.aserve.client:do-http-request "https://franz.com")
Process 5886 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.2
frame #0: 0x0000000102081090 libssl.3.dylib`SSL_ctrl
libssl.3.dylib`SSL_ctrl:
-> 0x102081090 <+0>: stp x20, x19, [sp, #-0x20]!
0x102081094 <+4>: stp x29, x30, [sp, #0x10]
0x102081098 <+8>: add x29, sp, #0x10
0x10208109c <+12>: mov x20, x2
(lldb) si
<<<hang here>>>
Again, it only started with macOS 15. We have not seen this on any previous version.
More detail:
$ codesign -vvvv /Applications/AllegroCL64express.app
/Applications/AllegroCL64express.app: valid on disk
/Applications/AllegroCL64express.app: satisfies its Designated Requirement
$
$ codesign -d --entitlements - /Applications/AllegroCL64express.app
Executable=/Applications/AllegroCL64express.app/Contents/MacOS/AllegroCL64express
[Dict]
[Key] com.apple.security.cs.allow-dyld-environment-variables
[Value]
[Bool] true
[Key] com.apple.security.cs.allow-jit
[Value]
[Bool] true
[Key] com.apple.security.cs.disable-library-validation
[Value]
[Bool] true
[Key] com.apple.security.get-task-allow
[Value]
[Bool] true
$
The other thing we noticed in debugging this is even though we set DYLD_LIBRARY_PATH, another libssl seemed to be found by lldb. For example, in this case 3 versions of SSL_new were found by lldb:
$ lldb alisp
(lldb) target create "alisp"
Current executable set to '/Applications/AllegroCL64express.app/Contents/Resources/alisp' (arm64).
(lldb) _regexp-env ACL_OPENSSL_VERSION=30
(lldb) _regexp-env DYLD_LIBRARY_PATH=/opt/homebrew/opt/openssl@3.0/lib:
(lldb) br s -n SSL_new
br s -n SSL_new
Breakpoint 1: 2 locations.
(lldb) run
Process 6339 launched: '/Applications/AllegroCL64express.app/Contents/Resources/alisp' (arm64)
Copyright (C) 1985-2023, Franz Inc., Lafayette, CA, USA. All Rights Reserved.
...
CL-USER(1): (progn (require :ssl)(require :aserve))
; Fast loading
; /Applications/AllegroCL64express.app/Contents/Resources/code/SSL.002
...
T
CL-USER(2): (net.aserve.client:do-http-request "https://franz.com")
Process 6339 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.3
frame #0: 0x00000001020803ec libssl.3.dylib`SSL_new
libssl.3.dylib`SSL_new:
-> 0x1020803ec <+0>: stp x20, x19, [sp, #-0x20]!
0x1020803f0 <+4>: stp x29, x30, [sp, #0x10]
0x1020803f4 <+8>: add x29, sp, #0x10
0x1020803f8 <+12>: cbz x0, 0x102080700 ; <+788>
(lldb) br list
Current breakpoints:
1: name = 'SSL_new', locations = 3, resolved = 3, hit count = 1
1.1: where = libboringssl.dylib`SSL_new, address = 0x0000000193f1b160, resolved, hit count = 0
1.2: where = libssl.48.dylib`SSL_new, address = 0x000000026907f64c, resolved, hit count = 0
1.3: where = libssl.3.dylib`SSL_new, address = 0x00000001020803ec, resolved, hit count = 1
(lldb)
We are out of ideas on how to debug this.
DTS doesn’t support OpenSSL. From our perspective it’s just like any other open source library. For definitive advice on how to use it, you should look to the library’s support channel.
However, there’s a bunch of history here, so I want to reply here to make sure you get on the right path.
Way back in the day, Apple included OpenSSL in the macOS SDK. That changed in macOS 10.7 (circa 2011) where we removed the OpenSSL headers and libraries from the SDK. The canonical way to use OpenSSL in your product is to build it from source and embed a copy in your app, just like other open source libraries.
Note We still include the OpenSSL runtime libraries in macOS, but those are present purely to maintain compatibility with existing binaries.
If you decide to embed OpenSSL in your app then I recommend that you give it an rpath-relative install name. See the following for more details on that:
-
Dynamic Library Identification DevForums post, and specifically the Dynamic Library Standard Setup for Apps post it links to
-
Embedding nonstandard code structures in a bundle documentation, which has a specific example of how to set this up using
install_name_tool
If you adopt this strategy and things still fail, please post a crash report of the failure. Again, we don’t support OpenSSL, but the crash report might be enough for me to offer you further hints.
See Posting a Crash Report for info on how to post a crash report.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"