Rendering drawingGroup while dragging

I have a custom SwiftUI view that renders something which is vastly larger than the viewable area, and allows the user to drag the view around to see different portions of it. My view is composed of a large number of paths (thousands) and therefore it has a drawingGroup() on it so that it performs reasonably. Unfortunately, when I add this drawingGroup() then view no longer renders portions of the view that were out-of-frame before dragging started. From what I've read, the drawingGroup is moving the rendering into an offscreen buffer, so I assume that the issue is that this buffer is not being updated during the gesture. Is there a way to have the offscreen buffer update during dragging, or, alternatively, oversize the buffer by 3x in each dimension so that dragging has a large enough field to scroll within?