Just to add some information:
When I examine the files that are stored via lfs in my local repository, their contents are like this:
version https://git-lfs.github.com/spec/v1
oid sha256:a864fa[..]f8dfc6c35c712
size 346287584
Am I using lfs wrong in the first place?
PS: I managed to check out the actual file from lfs via "git lfs checkout". So locally, everything is fine. Still the question remains how I can bring xcode to get access to the actual data