I'm using AppCenter and MSAL with SPM, and I managed to solve the issue.
First I tried the local package idea suggested above but that didn't solve anything.
AppCenter is rolling a fix (which at the moment of writing this isn't merged yet): https://github.com/microsoft/appcenter-sdk-apple/pull/2433
Their fix is to remove the hardcoded library type from their Package.swift.
For MSAL, they don't have this hardcoded setting, so I'm not sure what exactly causes the issue. But gladly on Xcode 14.1 beta 3 (14B5033e), using MSAL via SPM won't cause the error anymore.
So a solution for me was to pin AppCenter to that branch + use Xcode 14.1 beta.