For me the solution was to use .highPriorityGesture(drag) when attaching drag to inner view.
I have a joystick inside vertical scroll view. Problem was that joystick gesture was often hijacked by the scrollview when starting the drag gesture in a vertical direction. When it was started in horizontal it was working fine.