I've tried importing a Swift package from a private GitHub repository using a Personal Access Token (PAT), and it worked.
If you're not familiar with PATs, you can check out this resource for more information:
To use a PAT to access a private repository, you can insert it into the middle of the repository's URL like this:
- "https://<your-PAT-string>@github.com/<your-username>/<your-repo-name>.git"
I've only tried using the classic token to access a private repository, but I believe a fine-grained token with read-only access to repositories only should work as well, and is generally considered to be more secure.
Also, make sure you've created a version tag for your package. I couldn't find any other way to specify which version of the package to import, such as specifying a branch name or anything else I could do in Xcode.
I hope this information is helpful for you!