How to prevent auto-zooming with IOS MapKit when using userTrackingMode = .followWithHeading? (using SwiftUI)

How can one prevent auto-zooming with IOS MapKit when using userTrackingMode = .followWithHeading? That is I am setting the current user location to the centre of the screen, and have "view.userTrackingMode = .followWithHeading" so that the map orientates to north, but when you zoom in/out manually the MapView automatically overrides this and zooms back to the level it seems to prefer being at.

I was to be able to zoom in, then the zoom level stays like this, whilst it keeps the map centred to user location, and keeps auto-rotating to keep map aligned to north.

I am using SwiftUI so have effectively do have the location being passed into GCMapView as a parameter (as the means to keep the SwiftUI GCMapView up to date with latest user location). So not sure if this is causing an issue?

Some key bits (have pulled some code out to show relevant lines) of the MapKit call backs I'm using:

struct FlightView: View {
    @EnvironmentObject var locationManager : GCLocationManager
    @State var centreUserLocation : Bool = false

    var body: some View {
            GCMapView(
                flight: flight,
                userLocation: locationManager.userLocation,
                centreUserLocation: centreUserLocation,
                initalZoomDone: initalZoomDone
            )
    }
}

struct GCMapView : UIViewRepresentable {
    let map = MKMapView()
    func makeUIView(context: Context) -> MKMapView {
        map.delegate = context.coordinator
        map.isRotateEnabled = true
        map.userTrackingMode = .followWithHeading
        map.showsUserLocation = true
        return map
    }
    func updateUIView(_ view: MKMapView, context: Context) {
        if let userLocation = userLocation {
            view.centerCoordinate = userLocation
            view.userTrackingMode = .followWithHeading // Needed to keep map rotating to align to North
        }
    }
    
    func makeCoordinator() -> Coordinator {
        return Coordinator(self)
    }
    class Coordinator: NSObject, MKMapViewDelegate {
        var parent: GCMapView
        init(_ parent: GCMapView) {
            self.parent = parent
            super.init()
        }
    }
}