In my case, I create a custom button with a CAShapeLayer. I set the layer's path in layoutSubviews(), and then the bug appear.Just as you said, When I tap the button, it run to the textField's leftView, while the textField became the first responser, it run back the rightView.
So I state a var to control the layout, and the issue resolved.
private var layout = false
override func layoutSubviews() {
super.layoutSubviews()
//
if self.layout {
return
}
self.layout = true
//do layout
}