I have this exact question. I want to use a specific SDK, I don't want to modify piles of build scripts that I don't own.
By the way, xcode-select will be on your mac regardless of not having Xcode.app installed. It is irrelevant though, as it does not include a way to specific SDKs.