Internal error (errSSLInternal) on SSLHandshake

Hi, we're trying to setup a SSL connection over UDP between a client and a server on the Mac.


Therefor we're using the standard Security framework API. The SSL connection requires a client certificate, so that the server can verify the client. We've imported the CA certificate and private key in the keychain and then use`SecItemCopyMatching()` to retreive `SecItemRef` pointers to these keychain items. Then we create a `SSLCreateContext()` and use `SSLSetCertificate()` to set the CA and private key for the SSL context. Additionally we set the`kSSLSessionOptionBreakOnServerAuth` option to handle the server authentication ourselves (which is currently set to always pass). Of course, we also set up the context with a write/read function etc. After setting everything up, we start the handshake with the context `SSLHandshake(context)`. Then, after sending and receiving some data, we receive a `errSSLPeerAuthCompleted` error (for the server validation), which we process by simply calling `SSLHandshake(context)` again.


This is when things become weird. The handshake fails with `errSSLInternal`. So apparently during the client authentication something goes wrong internally and in the console app we see the following output:


default
14:15:58.370192 +0100
App
CSSM Exception: -2147416017 CSSMERR_CSP_ACL_ENTRY_TAG_NOT_FOUND
default
14:15:58.372311 +0100
App
CSSM Exception: -2147416017 CSSMERR_CSP_ACL_ENTRY_TAG_NOT_FOUND
default
14:15:58.374485 +0100
App
caught CssmError: -2147416017 CSSMERR_CSP_ACL_ENTRY_TAG_NOT_FOUND
default
14:15:58.377735 +0100
App
CSSM Exception: 32 CSSM_ERRCODE_OPERATION_AUTH_DENIED
default
14:15:58.380106 +0100
App
CSSM Exception: -2147416032 CSSMERR_CSP_OPERATION_AUTH_DENIED


We're unsure if this is related and we have no idea what this actually means. Do you guys maybe have any hints regarding this issue?


Thanks!

Up vote post of maximilianfrommunich Down vote post of maximilianfrommunich
2k views
Posted by
maximilianfrommunich
Reply
Add a Comment

Replies

CSSMERR_CSP_OPERATION_AUTH_DENIED
usually means that there’s something wrong with the key’s ACL in the keychain. You can learn more about this on this thread.

IMPORTANT The resolution of that thread was to code sign the calling code (using either Mac App Store or Developer ID signing). Make sure you’re doing that before you do anything else.

Beyond that, my next suggestion is that you try to break up the problem into parts. There’s three parts I can see:

  • The digital identity itself

  • Your import code

  • Secure Transport

You can check the digital identity with the TLSTool sample code:

  1. Import the digital using Keychain Access; make sure to set the ACL to ‘wide open’

  2. Set up a dummy TLS server using 

    TLSTool
    ; use the 
    -authenticate require
    option to force it to request a certificate from the client and the 
    -noverify
    option to allow any such certificate

  3. Try to connect to that server using 

    TLSTool

If that works then you know there’s nothing fundamentally wrong with your digital identity.

Note I realise that 

TLSTool
using TLS over TCP, and that you care about UDP. However, from a credentials perspective there’s no difference between TLS and DTLS.

Once you get that working you can then integrate your digital identity import code into the 

TLSTool
, which allows you to test that the import code is working as expected.

Finally, you get to deal with the glory that is Secure Transport (-:

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware 

let myEmail = "eskimo" + "1" + "@apple.com"
Posted by
eskimo
Post not yet marked as solved Up vote reply of eskimo Down vote reply of eskimo
Add a Comment