This is a known issue, and it comes down to the implementation of TabView in SwiftUI. The short version is that SwiftUI really wants to implement everything itself, but for various reasons they piggyback some things on UIKit. In this case they use UITabViewController to manage some parts of the logical ordering and view controller management, but ultimately do the drawing themselves using less resources. Each time you switch tabs it's actually using the same view and swapping out its content, discarding the View (a value type) it used before. When you switch back, that earlier state is gone, and gets recreated from scratch.
The long version is here: https://forums.developer.apple.com/thread/124749
I've also filed a bug report on this, and would encourage everyone to file duplicates: the more duplicates there are, the higher priority is given to this particular bug, and a higher priority generally leads to a quicker fix. My issue is FB7408323, and has the title "SwiftUI TabBar: tabbed content loses view state when switching tabs."