Center on screen two annotations while allowing MKCamera heading updates creates very jerky animations

In my app, I have a mode that rotates the MKMapView camera heading based on Core Location heading updates. This works great.

Code Block 
[UIView animateWithDuration:0.5 delay:0 options:UIViewAnimationOptionAllowUserInteraction animations:^{
   self.mapView.camera.heading = theHeading;//Comes from CLHeading
} completion:nil];

At the same time, I have two annotations, one is the current location, that I always want to keep on screen. So as the user gets closer to one of the annotations, the map will continue to zoom in.

Code Block 
MKMapRect flyTo = MKMapRectNull;
if (location) {
    MKMapPoint annotationPoint = MKMapPointForCoordinate(location.coordinate);
    MKMapRect pointRect = MKMapRectMake(annotationPoint.x, annotationPoint.y, 0, 0);
    if (MKMapRectIsNull(flyTo)) {
        flyTo = pointRect;
    } else {
        flyTo = MKMapRectUnion(flyTo, pointRect);
    }
}
if (CLLocationCoordinate2DIsValid(self.selectedCoordinate)) {
    MKMapPoint annotationPoint = MKMapPointForCoordinate(self.selectedCoordinate);
    MKMapRect pointRect = MKMapRectMake(annotationPoint.x, annotationPoint.y, 0, 0);
    if (MKMapRectIsNull(flyTo)) {
        flyTo = pointRect;
    } else {
        flyTo = MKMapRectUnion(flyTo, pointRect);
    }
}
[self.mapView setVisibleMapRect:flyTo edgePadding:UIEdgeInsetsMake(30, 30, 30, 30) animated:animated];

The Problem

The big issue here, is that as the user rotates the device, and the heading is updated, the visibleMapRect is also updated to keep both annotations on the map. This causes the map to animate back to "north up" and then quickly back to rotating the map camera. This creates a very jerky movement which is not what I want.

Is it possible to set the visibleMapRect and the camera heading at the same time without this negative side effect?