Downsides of using VMWare ESXi instances as Jenkins/Fastlane/Xcode build nodes?

Hi,

Our CI pipeline has 2013 Mac Pros running two Catalina VMs each using VMWare ESXi. Each VM is currently configured with macOS Catalina 10.15.5, Xcode 11.5, Jenkins, and Fastlane. Each VM is allotted exactly half the CPU cores and RAM of the host machine.

We've been experiencing intermittent build failures where some tests take too long to run and timeout, and sometimes network requests to fetch provisioning profiles also fail.

I'm wondering if perhaps we should reduce the CPU & RAM allotment to each VM in order to increase the resources available to the underlying host?

Or does anyone know of any fundamental flaws with this kind of setup that might contribute to instability? If so is there any other supported way to handle multiple simultaneous builds per machine?

We're just trying to take maximal advantage of the hardware available on our pipeline.

Thanks in advance for any tips/advice!

(Please see this post for details on the kinds of build issues we've been seeing.)