Apple support helped me with a workaround:
Create a new throwaway bundle ID
Restore your deleted app and refresh the page, then change the bundle ID to the throwaway bundle ID
Delete the app again, and you should be able to remove the original identifier that you wanted to remove
NOTE: you should still revoke all the original provisioning profile.