I'm on iOS 13 and Xcode 11, so it may be different. I haven't seen the issues around navigation links described in the OP.
Overall, there's a couple of options as a starting point to play with:
https://github.com/siteline/SwiftUIRefresh uses UIKit to provide a fairly simple way of controlling the refresh action (but sounds like the approach described in the OP that creates problems with navigation links)
https://gist.github.com/swiftui-lab/3de557a513fbdb2d8fced41e40347e01 is a pure SwiftUI implementation, but doesn't work on some views (including a list view in NavigationView)
Richiec
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution