Not sure if this solution works for everyone, but I found in my case that xcodebuild was reaching out because I had an account defined in Xcode > Settings > Accounts. Once I deleted my account from Xcode, xcodebuild sped up to the same speed as when I unplug my network cable.
But the caveat is that my team's build system doesn't have the "Automatically manage signing" checkbox checked for our projects and is setup to separately manage our certs / provisioning profiles / team identifiers, so removing my account from Xcode isn't a big deal.
aaclarke
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution