So I found out clang can do multiarch compiles (-arch arm64 -arch x86_64). But Apple seems to have left precompiled header support out. So I built the pch separately for each arch. That all works.
The next problem is that one needs to specify -include-pch foo.x64.pch and -include-pch foo.arm64.pch on the command line. This doesn't work on the compile line, since it tries to prepend arm64 AST to a x64 .o file, and vice versa.
So there is -Xarch_arm64 <arg> and -Xarch_x86_64 <arg>. But that option is limited to one argument. But "-include-pch foo.x64.pch" is two arguments.
More details of failed attempts here: https://github.com/llvm/llvm-project/issues/114626
And no splitting out the builds isn't the same, because then -valid_arch I don't think skips the other build. This are all libraries being built by Make, and then the universal app built using an Xcode project from the libraries.