SecIdentityRef without importing (SecPKCS12Import) into the keychain

How can I get an SecIdentityRef without adding to the keychain?


Running a secure web server using CocoaAsyncSocket requires an array of certificates where the first item is a SecIdentityRef. (The 2nd item is a SecCertificateRef which I can succesfully obtain using SecCertificateCreateWithData from my .pfx file containing the public and private keys).


The examples I have seen add the certificate to the keychain (using SecPKCS12Import) in order to get a SecIdentityRef, but I don't want to modify the keychain at all. (Note: my certificate is trusted by a root certificate which is already in the keychain).


Any advice is welcome. Thanks.

What platform are you on? On iOS

SecPKCS12Import
never imports the identity into the keychain; you have to explicitly call
SecItemAdd
to do that. So, the answer to your question:

How can I get an SecIdentityRef without adding to the keychain?

is, use

SecPKCS12Import
.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thank you for responding. Unfortunataley I am on Mac OS. The underlying requirement is to provide the correct data for the Security framework's SSLSetCertificate method.


OSStatus SSLSetCertificate ( SSLContextRefcontext, CFArrayRefcertRefs );

context: An SSL session context reference.

certRefs: 
The certificates to set. This array contains items of type
SecCertificateRef
, except for
certRefs[0]
, which is of type
SecIdentityRef
.


Do I have any alternative options? Can I generate the secIdentityRef from the private key and certificate which I already have?

Unfortunataley I am on Mac OS.

I wouldn’t say that was unfortunate (-:

Modern versions of OS X also allow you to import a PKCS#12 without putting it in a keychain. I’m not sure if

SecPKCS12Import
will do the trick on OS X but, failing that, you can use
SecItemImport
and pass nil to the
importKeychain
parameter.

Can I generate the secIdentityRef from the private key and certificate which I already have?

Not easily. Where did this private key come from? There’s really two options here:

  • You generated the private key on the Mac, in which case it’s probably already in the keychain.

  • You are getting the private key from off the Mac, in which case the easiest (and safest) option is to have the sender put it in PKCS#12 format.

Another option is to simply have your app manage its own private keychain, which means you can have this stuff in the keychain without messing up the user’s state.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I'm running into the same problem. I have a .p12 file that has both a certificate and the associated private key. If I use SecPKCS12Import() to extract the contents of the file, I get both the certificate and the key but they are added to the keychain (not my desired outcome, I just want the key). If I use SecItemImport(), the data is extracted and not added to the keychain but only the certificate is returned in the list of items, not the private key. Here's code running on OS X 10.11.3 that demonstrates the issue:

NSString *securityErrorMessageString(OSStatus status) { return (NSString *)SecCopyErrorMessageString(status, NULL); }

+ (SecKeyRef)privateKeyUsingSecPKCS12ImportFromP12File:(NSString *)filePath password:(NSString *)password {
        SecKeyRef key = NULL;
        NSData *p12Data = [NSData dataWithContentsOfFile:filePath.stringByExpandingTildeInPath];
        CFArrayRef items = CFArrayCreate(NULL, 0, 0, NULL);
        NSMutableDictionary *options = [NSMutableDictionary dictionary];
        if (password && password.length) [options setObject:password forKey:(id)kSecImportExportPassphrase];
        OSStatus status = SecPKCS12Import((CFDataRef)p12Data,
                                          (CFDictionaryRef)options,
                                          &items);
       if ((status == errSecSuccess) && (CFArrayGetCount(items) > 0)) {
             CFDictionaryRef identityDict = CFArrayGetValueAtIndex(items, 0);
             SecIdentityRef identity = (SecIdentityRef)CFDictionaryGetValue(identityDict, kSecImportItemIdentity);
             status = SecIdentityCopyPrivateKey(identity, &key);      
             if (status != errSecSuccess) {
                   NSLog(@"Could not copy private key from P12 file \"%@\": [%d] %@",
                          filePath, (int)status, securityErrorMessageString(status));
                   key = NULL;
             }
       } else if (status != errSecSuccess) {
             NSLog(@"Could not import items from P12 file \"%@\": [%d] %@",
                     filePath, (int)status, securityErrorMessageString(status));
       }
       CFRelease(items);
       return key;
}

+ (SecKeyRef)privateKeyUsingSecItemImportFromP12File:(NSString *)filePath password:(NSString *)password {
       SecKeyRef key = NULL;
       NSData *p12Data = [NSData dataWithContentsOfFile:filePath.stringByExpandingTildeInPath];
       CFArrayRef items = CFArrayCreate(NULL, 0, 0, NULL);
       SecItemImportExportKeyParameters parameters;
       parameters.version = SEC_KEY_IMPORT_EXPORT_PARAMS_VERSION;
       parameters.flags = ((!password || !password.length) ? kSecKeySecurePassphrase : 0);
       parameters.passphrase = (CFStringRef)password;
       parameters.alertTitle = (CFStringRef)@"Import Private Key";
       parameters.alertPrompt = (CFStringRef)[NSString stringWithFormat:@"Please enter the password for \"%@\".",
                                                                       filePath.lastPathComponent];
       parameters.accessRef = NULL;
       parameters.keyUsage = NULL;
       parameters.keyAttributes = NULL;
       SecExternalFormat inputFormat = kSecFormatPKCS12;
       SecExternalItemType itemType = kSecItemTypeAggregate;
       OSStatus status = SecItemImport((CFDataRef)p12Data,
                                       (CFStringRef)filePath.pathExtension,
                                       &inputFormat,
                                       &itemType,
                                       0,
                                       &parameters,
                                       NULL,
                                       &items);
       if ((status == errSecSuccess) && (CFArrayGetCount(items) > 0)) {
             if (CFArrayGetCount(items) == 1)
                   NSLog(@"Could not copy private key from P12 file \"%@\": only one item was extracted and it is: %@",
                                                      filePath, (id)CFArrayGetValueAtIndex(items, 0));
             // items always returns a single item array with the SecCertificateRef
             // and not the SecKeyRef for the private key in the p12 archive
             // otherwise, we'd extract it here
        } else if (status != errSecSuccess) {
              NSLog(@"Could not import items from P12 file \"%@\": [%d] %@",
                      filePath, (int)status, securityErrorMessageString(status));
        }
        CFRelease(items);
        return key;
}


Any help to etract the private key from the .p12 archive file without also adding the items to the keychain on OS X would be most appreciated.


Thanks,

Jon

If I use SecItemImport(), the data is extracted and not added to the keychain but only the certificate is returned in the list of items, not the private key.

Indeed. OS X definitely has the ability to deal with private keys that aren’t in the keychain (for example, if you convert the PKCS#12 to a PEM and import it using

SecItemImport
, you get back both the certificate and the private key, with the private key not in the keychain) but I also see the same problem as you do in the PKCS#12 case. I suspect that there is some way to make this work but, alas, I’ve run out of time to investigate this in the context of DevForums. I recommend you open a DTS tech support incident, which will allow me to allocate more time to this problem.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks for the help, Quinn. Ultimately, my solution, as you suggested, was just to create a temporary

SecKeychainRef
(
SecKeychainCreate
), import to that temporary keychain using
SecPKCS12Import
, extract the
SecKeyRef
from the resulting
SecIdentityRef
, then delete the
SecKeychainRef
(
SecKeychainDelete
). Not as elegant as I'd like but it works to just extract the private key that I need. This doesn't touch the default keychain and if the identity is already in that keychain, it remains there after this is done.

OK.

One thing to double check with your code is that you don’t mess up the keychain search list (as returned by

SecKeychainCopySearchList
). Back in the day various keychain functions would modify that search list implicitly. I think we fixed that, but I definitely recommend a unit test that checks that the search list remains constant across your keychain operations.

btw I did dig into the ‘

SecItemImport
returns just the certificate’ failure (some issues just keep preying on my mind) and discovered some interesting factoids.
SecItemImport
really does support importing private keys without putting them in the keychain, and that code all runs and works in the PKCS#12 case; internally I see both the certificate and the private key extracted from the PKCS#12. The problem arises when the code tries to match up the certificate and private key to form an identity. That code is failing in the no-keychain case, so you end up getting back just the certificate.

Notably, in the PEM case no matching occurs and thus you get back both the certificate and the private key.

This is clearly a bug and I’ve filed it as such (r. 25,140,029).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks again for the follow-up and for filing the bug. I don't think the keychain search list is getting messed up by my code. For completeness, here's the code I am using including a search for a private key after creating and deleting a new keychain to hold the imported credentials:

#import <Foundation/Foundation.h>
#import <Security/Security.h>

NSString *securityErrorMessageString(OSStatus status) { return (__bridge NSString *)SecCopyErrorMessageString(status, NULL); }

SecKeyRef privateKeyFromKeychain(NSString *keyName) {
     NSDictionary *query = [NSDictionary dictionaryWithObjectsAndKeys:
                                                  (id)kSecClassKey, (id)kSecClass,
                                                  (id)kSecAttrKeyClassPrivate, (id)kSecAttrKeyClass,
                                                  keyName, (id)kSecAttrLabel,
                                                  (id)kCFBooleanTrue, (id)kSecReturnRef,
                                                  nil];
     SecKeyRef key = NULL;
     OSStatus status = SecItemCopyMatching((__bridge CFDictionaryRef)query, (CFTypeRef *)&key);
     if (status != errSecSuccess) {
          NSLog(@"Could not get private key with name \"%@\": [%d] %@", keyName, (int)status, securityErrorMessageString(status));
          if (key) CFRelease(key);
     }
     return key; //caller must use CFRelease on returned key
}

SecKeyRef privateKeyFromP12File(NSString *filePath, NSString *password, BOOL addIdentityToKeychain) {
     SecKeyRef key = NULL;
     NSString *p12Path = filePath.stringByExpandingTildeInPath;
     if (!password || !password.length) {
          NSLog(@"Password required for file: \"%@\".", p12Path);
          return key;
     }
     NSData *p12Data = [NSData dataWithContentsOfFile:p12Path];
     if (!p12Data) {
          NSLog(@"Could not read p12 data from file: \"%@\".", p12Path);
          return key;
     }
     OSStatus status;
     SecKeychainRef keychain = NULL;
     if (!addIdentityToKeychain) {
          //SecPKCS12Import will automatically add the items to the keychain
          //so create a new keychain for import then delete it
          //make sure we create a unique keychain name:
          NSString *temporaryDirectory = NSTemporaryDirectory();
          NSString *keychainPath = [[temporaryDirectory stringByAppendingPathComponent:[[NSUUID UUID] UUIDString]] stringByAppendingPathExtension:@"keychain"];
          status = SecKeychainCreate(keychainPath.UTF8String, (UInt32)password.length, password.UTF8String, FALSE, NULL, &keychain);
          if (status != errSecSuccess) {
               if (keychain) {
                    SecKeychainDelete(keychain);
                    CFRelease(keychain);
               }
               NSLog(@"Could not create temporary keychain \"%@\": [%d] %@", keychainPath, (int)status, securityErrorMessageString(status));
               return key;
          }
     }
     NSMutableDictionary *options = [NSMutableDictionary dictionary];
     [options setObject:password forKey:(id)kSecImportExportPassphrase];
     if (!addIdentityToKeychain) [options setObject:(__bridge id)keychain forKey:(id)kSecImportExportKeychain];
     CFArrayRef items = CFArrayCreate(NULL, 0, 0, NULL);
     status = SecPKCS12Import((CFDataRef)p12Data, (CFDictionaryRef)options, &items);
     if ((status == errSecSuccess) && (CFArrayGetCount(items) > 0)) {
          CFDictionaryRef identityDict = CFArrayGetValueAtIndex(items, 0);
          SecIdentityRef identity = (SecIdentityRef)CFDictionaryGetValue(identityDict, kSecImportItemIdentity);
          status = SecIdentityCopyPrivateKey(identity, &key);
          if (status != errSecSuccess) {
               NSLog(@"Could not copy private key from P12 file \"%@\": [%d] %@", filePath, (int)status, securityErrorMessageString(status));
               if (key) CFRelease(key);
          }
     } else if (status != errSecSuccess) {
          NSLog(@"Could not import items from P12 file \"%@\": [%d] %@", filePath, (int)status, securityErrorMessageString(status));
     }
     if (keychain) {
          SecKeychainDelete(keychain);
          CFRelease(keychain);
     }
     CFRelease(items);
     return key; //caller must use CFRelease on returned key
}

int main(int argc, const char * argv[]) {
      @autoreleasepool {

             NSString *p12Path = @"path/to/file.p12";
             NSString *password = @"superSecret123";
             BOOL addIdentityToKeychain = NO;
             SecKeyRef privateKey = privateKeyFromP12File(p12Path, password, addIdentityToKeychain);
             NSLog(@"privateKeyFromP12File: %@", (__bridge id)privateKey);
             if (privateKey) CFRelease(privateKey);

             NSString *keyName = @"My Private Key";
             privateKey = privateKeyFromKeychain(keyName);
             NSLog(@"privateKeyFromKeychain: %@", (__bridge id)privateKey);
             if (privateKey) CFRelease(privateKey);

      }
       return 0;
}


Regards, Jon

I am not sure whether this bug (r. 25140029) has been fixed.

It has not. However, there are two workarounds discussed on this thread (using a PEM and using a custom keychain) and AFAIK those both continue to work.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Using a custom keychain is deprecated, and it seems nearly all APIs from the Security framework expect at some point for things to go into a keychain, which doesn't work for a macOS based CLI tool meant to run in continuous integration on Linux or macOS.

I've resorted to using a private API to created a SecIdentity from a SecCertificate and a SecKey that I already have in memory.

It feels very unfortunate that I've had to do this to implement mTLS while using Foundation, because otherwise I would have to abandon Foundation (for cURL or SwiftNIO) completely simply because of this missing private API. Can Apple please expose this API officially in a future Security framework release?

Also for reference, I am not the only one using this API, I learned of it from major software projects using it as well.

Can Apple please expose this API officially in a future Security framework release?

The best way to get that request in front of the folks who have the power to enact change is to put it in enhancement request. Please post your bug number, just for the record.

Using a custom keychain is deprecated

True. But then again the file-based keychain is kinda deprecated as whole (see TN3137). Sadly, there are still places where you have to use it.

and it seems nearly all APIs from the Security framework expect at some point for things to go into a keychain, which doesn't work for a macOS based CLI tool meant to run in continuous integration on Linux or macOS.

I don’t understand how Linux factors into this because the Security framework isn’t present there at all.

On the macOS front, there’s nothing stopping a command-line tool running on a CI machine using the keychain. There are some pain points but no showstoppers. This is something I touch on in Resolving errSecInternalComponent errors during code signing.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

SecIdentityRef without importing (SecPKCS12Import) into the keychain
 
 
Q