This issue did not occur until we mandated our workforce update to 17.2 to be compliant. Once users who had large contact lists updated, they started to experience delayed / freezing calls.
We utilize Microsoft Intune as our MDM, but I have to assume this is hitting other MDM's as well since this is a policy that the management profile portion of the software either has toggled on or off. The policy triggering this issue, if enabled is "Block viewing corporate documents in unmanaged apps".
I've linked a dummy google account with an imported contact list of 16k contacts (yes somehow we have a handful of folks who have this) and the issue replicates with this policy enabled.
With a mirrored configuration profile, minus this setting enabled, calls flow as they should. Even with the large contact list.
I have no clue if this is right place for this, but the issue still persists after updating my device to 17.3