Important item in Keychain seems to have disappeared (after years)

I’m presuming this is on the Mac. That’s important, because the keychain is much more complex on the Mac than it is on iOS. See TN3137 On Mac keychain APIs and implementations.

I have some generic advice about how to use the keychain effectively:

• SecItem: Fundamentals

• SecItem: Pitfalls and Best Practices

Looking at your code, I don’t see anything fundamentally wrong with it.

The code doesn’t opt in to the data protection keychain, so it’s gonna be using the file-based keychain. Moreover, it’s likely that you created the item in the default keychain, which is usually the login keychain. Given that, you should be able to look for the keychain item using Keychain Access. That’ll tell you whether it’s there or not.

If it’s not, you can’t fix that with code. You’ll have to find a keychain with the item in it. I have hints and tips on how to do that, for very different reasons, in The Care and Feeding of Developer ID.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Yes, it's on macOS. I'm trying to piece together exactly what code I was running at the time, a few years ago, but I think the code that created the item may have used the data protection keychain, which you mentioned.

var query: [String: Any] = [kSecClass as String: kSecClassGenericPassword,
                            kSecAttrAccount as String: account,
                            kSecUseDataProtectionKeychain as String: true,
                            kSecValueData as String: password]
if let label = label {
    query[kSecAttrLabel as String] = label
}
let status = SecItemAdd(query as CFDictionary, nil)

A quick test suggests that even if I add it like that, the snippet in my original post, with SecItemCopyMatching, finds the item that was created, even though it doesn't have that kSecUseDataProtectionKeychain key in its query.

A second piece of info: in ~/Library/Keychains, there is a file named login_renamed_1.keychain-db from 2022. I don't know what creates a file like this, but I wonder if it's from a password reset/change that I did on my MacBook. When I try to open it in Keychain Access, it appears in the left nav, but I don't think it's completely open / searchable, because I can't "unlock" it. It's not accepting my current MacBook password. Does that mean it is locked by my previous MacBook password and might contain the missing password item?

If so, it's going to be very difficult for me to remember that password, but maybe not impossible. I almost remember it - it was a certain word or two with a few extra characters thrown in. That might be "crackable", unlike the random 256 bit key I'm trying to find.

Right. If you don’t supply a value for the kSecUseDataProtectionKeychain attribute [1], SecItemCopyMatching will query both keychain implementations. And if you request multiple results via kSecMatchLimit, it can return results from different keychain implementations.

Possibly. The renamed label means that at some point macOS renamed on old keychain out of the way because it wasn’t usable for some reason. That could be because it didn’t have the unlock password, but it’s also possible there’s some other problem in play (like a file corruption issue).

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] For the older kSecAttrSynchronizable attribute, there is kSecAttrSynchronizableAny.

I have a few sub-questions that would help me narrow down my search.

1. Can I see every item name in login_renamed_1, without unlocking?

This could help me skip the large effort I'm expending trying to crack my old password. When I open that old login_renamed_1.keychain-db file in Keychain Access, it appears in the left nav under "Custom Keychains" and it is locked. However, I can still click on it and see the names of the items in the main window area to the right. My lost master key item is not there.

Would unlocking it reveal more names, more items? If not, if I can already see the entire content of that keychain, then I don't see a point in continuing my current CPU/GPU-heavy attempts to recover my partially-remembered old laptop password, with the goal of unlocking this keychain.

2. Is the entire data protection keychain in the file keychain-2.db?

One of the links you (Quinn) posted says that the data protection keychain is here, but people say "file-based keychain" to refer to the other (non data protection) keychain. This is confusing me, since this looks to be "file based" as well, if it is contained in the SQLite DB in keychain-2.db file?

3. Could the update to Passwords app have deleted or hidden my SecItem?

Data migrations are dangerous. At some point after I originally called SecAddItem in late 2020, Apple introduced the Passwords app and passwords seemed to move there. Are there any known issues here with SecItems becoming hidden, or lost?

4. Can I query the names of SecItems in keychain-2.db, using sqlite3?

If my item is somehow lost in that database file, I wonder if I might recover it with some manual SQL selects. I can use sqlite3 to view the file contents, but most of the table contents appear... cryptic.