Huge difference in complier performance on identical Macs

My company as a large iOS Xcode Project (>9000 source files.) We do Continious Integration testing/publishing on a pool of Mac Mini Servers. We were thrilled to see the increase in performance in the new late 2018 Mac Minis; however, we have a situation that is hard to understand. We purchasd 6 identically configured new (late 2018) Mac Minis and set them up them identically using the same script for all machines. One, and only one, of the new Macs runs the Xcode compiler much faster than the others. Certain xcodebuild operations run 5X faster on this one Mac!


Obviously we want to get this great performance on all our new Macs. We tried to clone this Mac to one of the others but did not see any change in performance. We also did tests to ensure the same operating tempature and environment was present for the fast and slower Macs. Running Intel Power Gadget on the different machines reveals that the slower Macs do not reach the same level utilization when compliing a large batch of .m files. The fast Mac reaches an average of around 99% in this process while the others only get to about 82%.


Does anyone have an idea how to determine why this one Mac has such better complier performance than the others? How can we get the other identical Macs to get the same performance?


All Macs are configured as follows:

macOS 10.14.4

Xcode 10.2 Build Version 10E125

Processor: 3.2 GHz Intel Core i7

Memory: 32 GB 2667 MHz DDR4

Accepted Reply

The issue was that CrowdStrike Falcon anti-virus was running. Turning this off resulted in the faster speed on the slow machines.

Replies

I'm not familiar w/IPG...does it do SSD speed checks too? Have you done comparative drive checks on all of those minis?


As an exercise, I'd move the ram from the top performer to another mini and see if the performance changes on either one. The 16 steps required may make such a test non-trivial, tho, so...

Just a guess.


Have you looked at how your files are stored on the various servers ? Are they replicated on each server or localized on various servers ? Maybe the fast Mac has some essentials files stored locally.

Thanks for this reply. All the files are brought down locally prior to starting the compile. We looked at the disks and they seem to be the similar in terms of free space. So I don't think this is it.

If that's possible, did you tyry disconnecting a slower Mac from the network ? Could it be that the build searches, in fact, for files on other servers, even though you have a local copy ?

The issue was that CrowdStrike Falcon anti-virus was running. Turning this off resulted in the faster speed on the slow machines.