I struggled with this one for a while too. I was eventually able to get passkeys working in the simulator using locally generated certificates. Here are a few things I did differently to you (but I'm not 100% sure which one would make the difference in your case),
hosting the server on port 443 (and not including the port in Associated Domains settings in Xcode)
generating a Root CA and leaf certificate chain, and trusting the generated Root CA in the simulator
jdoxey
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution
Last seen
User for