Keychain item disappears after system sleep - macOS 10.15 only

Hi there,


We have a Developer ID app that has been functioning normally for well over 10 years, we store the username and password of their account to the local Keychain very simply:


NSURLCredential *credential = nil;

NSURLProtectionSpace *space = [[NSURLProtectionSpace alloc] initWithHost:[myUrl host]
  port:[[myUrl port] integerValue]
  protocol:[myUrl scheme]
  realm:nil
  authenticationMethod:NSURLAuthenticationMethodHTMLForm];

credential = [[NSURLCredential alloc] initWithUser:[self userid]
   password:[self password]
    persistence:NSURLCredentialPersistencePermanent];

[[NSURLCredentialStorage sharedCredentialStorage] setCredential:credential forProtectionSpace:space];


We've been able to reproduce on 10.15 machines that after some amount of time, and especially after system sleep, the keychain item simply dissappears. It's not just that the app can't access it, it simply is gone from Keychain Access entirely. In fact if you leave Keychain Access open, then wake the system from sleep, you can literally see the item dissappear before your eyes if that window is in the foreground.


Note that the app can launch/quit/launch/quit many times and read back the item (so no permission problem), but then "randomly" the item will just dissappear, doesn't matter if the app is running or not. I thought maybe the underlying implementation changed on Catalina to support the new iOS-style Keychain and we don't have a provisioning profile or anything, so I re-wrote the code to first use SecItemAdd, same problem, then the ancient SecKeychainAddInternetPassword API, still dissappears.


I even had it dirty the keychain item (by re-setting the password data and then seeing the date modified change in Keychain Access) each launch time to see if it was somehow getting stale (though I don't know how), doesn't make a difference.


This only happens on 10.15, so I'm not sure what else to try. Any idea why an Internet Password Keychain item would just be removed from the system? My test account does have iCloud Keychain turned on, but this dissappearing entry is only being written to/read from the "login" Keychain and shouldn't be affected by syncing. Especially when using SecKeychainAddInternetPassword is used.

Replies

I’m not aware of any obvious explanation for the behaviour you’re reporting. Two things:

  • If you put this code in a standalone test app, does the problem reproduce?

  • If you use a separate keychain class (say generic password), does the problem reproduce?

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks Quinn.


The results are in:


  • Standalone simple test app reproduces (I'll file a bug), the password was gone when I got to work this morning.
  • Generic password keychain class stayed persistent across sleep. So I'll migrate everyone to that for now and revisit someday.

Standalone simple test app reproduces … the password was gone when I got to work this morning.

Whoah, freaky!

I'll file a bug

Thanks! And please do post the bug number here. I’d love to try this myself.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Yeah, real freaky, especially since it's happening in the field too. Filed as FB7435783. I included the simplest sample project.


Thanks again for the suggestion to use the generic password.

Quinn,


Would this possibly be bug that would also appear on iOS?


I am experiencing something extremely similar in iOS 13.2.2 (not sure when it started but I believe 13.2). Our app relies on keychain items to load, and after a phone reboot, the keychain items are gone with an OS status of -25300. It affects both Generic Passwords and Certificates & Keys. It is not reproducible consistently. Usually we're able to reproduce it after putting our app in the background, taking bursts of hundreds of photos, then rebooting the phone, then re-loading the app through the search page.

Filed as FB7435783. I included the simplest sample project.

Thanks.

This is a weird bug. I ran your test project on a VM and I actually managed to reproduce the problem. However, sleep and wake are always a little weird in VMs, so I switched to a real Mac. I had the app installed on a victim Mac here in my office all day yesterday, periodically waking it up and sleeping it, and didn’t see the problem once.

Weird.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Would this possibly be bug that would also appear on iOS?

It’s possible, but the symptoms you’ve described don’t match the ones reported by tullera. In their case it only affects Internet passwords, and a switch to generic password avoids the problem.

It is not reproducible consistently. Usually we're able to reproduce it after putting our app in the background, taking bursts of hundreds of photos, then rebooting the phone, then re-loading the app through the search page.

Have you tried isolating that into a small test app? If so, and it reproduces there, that’s definitely bugworthy. Make sure to include:

  • A copy of your test app

  • A sysdiagnose log — This is vital for the keychain engineers to investigate the problem. You can learn more about sysdiagnose logs on our Bug Reporting > Profiles and Logs page.

  • A timeline — That is, rough timestamps for when you ran your app and created the keychain item, when you starting taking photos, when you restarted, and when you ran your app the second time and noticed the keychain item missing.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I'm glad that you were able to reproduce, I've only reproduced it in a VM at first, but was able to reproduce on a real Mac and our customers are on real machines as well.


Such a strange thing. I wonder if switching to using a provisioning profile and then using the iOS-style Keychain (kSecUseDataProtectionKeychain) would make a difference, I wasn't prepared to go down that road though, so this works for now, thanks again.

Thanks for your reply Quinn!


Yes, I was able to isolate the issue to a small app. Upon more troubleshooting, it appears that in iOS13.2 and on, some Core Bluetooth behavior changed. The presence of CBCentralManager CBCentralManagerOptionRestoreIdentifierKey launches the app before the user unlocks their phone with a passcode for the first time after a reboot. Because of this, all data (keychain, UserDefaults, etc) is unavailable.


I have filed FB7447530 with the sample app, and a lot more info. Not sure if it's a bug or expected behavior, but I think bug because launching the app before first unlock can cause a lot of issues if the data that the app depends on is not available. And we've never seen this behavior where the app launches before first unlock in any of the previous iOS versions we have been using (all the way from iOS10). Also, iBeacons does not wake up the app before first unlock, and seem to properly wait until user unlocks the phone for the first time.


Thanks again!