Hello! I have the same problem, but I figure out how to turn off the fallback behavior:
Just delete is_arm_mac call in condition here https://github.com/keras-team/keras/blob/5849a0953a644bd6af51b672b32a235510d4f43d/keras/optimizers/__init__.py#LL301C9-L301C9
Of course, it is a temporary and clumsy solution, I'd like to contribute with a proper fix. Here is an issue in Keras repo: https://github.com/keras-team/keras/issues/18224
abogomazov
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution