I could solve the problem by opening the created file in Quicktime and re-exporting it from there again.
I have similar problems using XCode 9.4.1 (and XCode 9.1)
Horrible artefacts making the feature totally useless.
I checked Xcode10 beta 6, and while the compression artefacts are not there anymore, most of the frames in the video are out of place (some appears too soon or too late).
I'll try with XCode 9.0 since I also remember that it used to work, but perhaps it is a problem with macos or some quicktime codecs. I have no idea.