So I set up the most minimal project I could exhibiting the behaviour and it still happens. For some reason it took me a whole day to realise a workaround was to just keep track of the draw rectangle myself, overriding the setNeedsDisplay functions to store it in my own variable. Works lightning fast now :) var actualDrawRect : CGRect? override func setNeedsDisplay(_ rect : CGRect) { super.setNeedsDisplay(rect)     if actualDrawRect == nil {         actualDrawRect = rect     } else {         actualDrawRect = actualDrawRect!.union(rect)     } }      override func setNeedsDisplay() {      super.setNeedsDisplay()     actualDrawRect = CGRect(origin: .zero, size: bounds.size) }