SwiftUI and frequent state updates for animation

I have a use case where I want to animate a sort of progress view based on the current playback position of an audio playback node. I draw the view myself using primitive shapes (in case that matters).

Let's say our view consists of two rectangles:

Code Block 
struct ProgressView: View {
  let progress: CGFloat
  var body: some View {
    GeometryReader { g in
      HStack(spacing: 0) {
        Rectangle().fill(Color.red)
          .frame(width: g.size.width * self.progress, height: g.size.height)
        Rectangle().fill(Color.blue)
          .frame(width: g.size.width * (1-self.progress), height: g.size.height)
      }
    }
  }
}

In a different class, I have the following code (simplified):

Code Block 
class Conductor {
	@Published var progress: Double = 0
  func play() {
		self.player.play()
    self.playbackTimer = Timer.scheduledTimer(withTimeInterval: 0.05, repeats: true) { _ in
      self.progress = self.player.currentTime / self.totalTime
    }
  }
}

Then I can update my view above as follows:

Code Block 
struct UpdatedProgressView: View {
	@EnvironmentObject private var conductor: Conductor
	var body: some View {
		ProgressView(progress: $conductor.progress)
	}
}

This works (assuming I have no typos in the example code), but it's very inefficient. At this point, SwiftUI has to redraw my ProgressView at 20Hz. In reality, my progress view is not just two Rectangles but a more complex shape (a waveform visualisation), and as a result, this simple playback costs 40% CPU time.

It doesn't make a difference whether I use drawingGroup() or not. Then again, I'm quite certain this is not the way it's supposed to be done. I'm not using any animation primitives here, and as far as I understand it, the system has to redraw the entire ProgressView every single time, even though it's just a tiny amount of pixels that actually changed.

Any hints on how I should change my code to make it more efficient with SwiftUI?