After trying multiple combinations, what worked for me (in Sonoma 14.5) was granting permission in Privacy & Security > Accessibility to all of the following:
assistant_service in /System/Library/PrivateFrameworks/AssistantServices.framework/Versions/A/Support/assistant_service
Shortcuts
Siri
siriactionsd in /System/Library/PrivateFrameworks/VoiceShortcuts.framework/Versions/A/Support/siriactionsd