This issue has been occurring for my team for months, typically from a private Reposilite instance for hosting some private packages, but occasionally other public packages as well. The Reposilite server is used by several teams with different build systems, and Xcode Cloud is the only one failing randomly like this. After much investigation where we whitelisted all Xcode Cloud IPs, added additional logging to the server, etc. all to no effect, I eventually found that I can resolve packages in the ci_post_clone.sh script, which ends up putting them in a cache that the later build step can find. I now do this in a retry loop, with redoing it 5 times with a small delay in between each attempt. This usually works, but we still get cases where it'll fail even that - once every couple of weeks or so. Rebuilding lets it work again.
Below is the relevant snippet from my ci_post_clone.sh script:
#!/bin/sh
function fail {
echo $1 >&2
exit 1
}
function retry {
local n=1
local max=5
local delay=15
while true; do
"$@" && break || {
if [[ $n -lt $max ]]; then
((n++))
echo "Command failed. Attempt $n/$max:"
sleep $delay;
else
fail "The command has failed after $n attempts."
fi
}
done
}
echo "Pre-resolving packages"
retry swift package resolve
Post
Replies
Boosts
Views
Activity
Saw the same here too. To clear it I had to drop down the version list and select the "Xcode 16.1 (16B40)" build again. If I had to guess, it has something to do with the "usage update" announcement - maybe there's a new "version" of that Xcode 16.1 release for Xcode Cloud.
¯_(ツ)_/¯
Seeing the same thing here. Our worst build that I'm aware of took 1h 55 minutes in the queue before it started, then before running tests inside the build was paused for another hour while it apparently waited for a free machine to run the tests. Others are all over the map, some taking approximately normal or just slightly more, some taking 2+ hours.
The fact that the https://developer.apple.com/system-status/ page shows Xcode Cloud as green while it takes up to 3.5 hours to run what is normally a sub 30 minute build is ridiculous. I'm hoping this means that there are some infrastructure changes being done, but I won't hold my breath.
I've found a workaround for the issue that I'll be rolling out to the team soon - in short, move the package resolution up a step, and add retry logic.
I grabbed a retry function from a Stack Overflow post (which apparently I can't link here, but you could find it pretty easy searching for details from the function below), dropped it into the ci_post_clone.sh script along with a call to do package resolution.
Doing the package resolution here will cache the results so the later resolution done during the build phase doesn't need to go out to the internet and try to download them again. In my tests so far I've found that it usually resolves correctly by the 2nd or 3rd attempt.
#!/bin/sh
function fail {
echo $1 >&2
exit 1
}
function retry {
local n=1
local max=5
local delay=15
while true; do
"$@" && break || {
if [[ $n -lt $max ]]; then
((n++))
echo "Command failed. Attempt $n/$max:"
sleep $delay;
else
fail "The command has failed after $n attempts."
fi
}
done
}
echo "Pre-resolving packages"
retry swift package resolve
I'm seeing the same issue, though with a Reposilite server. Happened first about a month ago and it stopped pretty much all of our builds from working. It then just kind of resolved itself while I was looking into it. Now it just started erroring out again starting today, just randomly in some builds.
Not sure if this will help anyone else, but just in case I'm posting how I was able to address it when I ran into this issue. The root cause was a conflict between a version specified in Package.swift and the Package.resolved file. The Package.swift file had just been updated to use the package(url:exact:) call for the dependency instead of package(url:from:) after running into an unrelated issue where local builds were using a newer version of a dependency than was specified in the Package.resolved file (so, what was used in Xcode Cloud). Unfortunately, Xcode wouldn't update the Package.resolved file after doing this change for some reason, and that fact was missed when those changes were checked in. I had to quit Xcode and run swift package resolve in terminal to get the file updated. Once this was checked in, the error was resolved.
As a tangentially related side note - I was constantly running into differences between the Package.resolved file in the root of the project vs. the one inside the workspace. To get around this, the workspace version was added to the .gitignore, and then in Xcode Cloud's ci_post_clone.sh script I copy the root project into that location. Now I only have to worry about keeping one file up to date.