Loads of keychain dialogs that won't dismiss when building

I'm attempting to build and have lots of code sign dialogs pop up. I'm entering the correct password and selecting Always Allow (selecting just Allow doesn't make any different) but they won't dismiss, instead more appear.

I've not found a way of getting rid of them - all the buttons are greyed out. Restarting the desktop doesn't get rid of them. I have to restart the Mac to get rid of them.

Why won't they go away and how can I force them to go away?

(Sonoma 14.2.1 / Xcode 15.2)

Loads of keychain dialogs that won't dismiss when building
 
 
Q