How to get SwiftUI views to move, not fade when animating change

I am trying to get SwiftUI views to move from one position to another using animation. I know that when the identity of the views change this often result in a fade instead of an animation but I do not believe this is happening in my case.

Below is a simple example program with four views from a 2D array (the 2D array is important). There is a button to rotate the views clockwise.

I tried using two ForEach blocks but I cannot use it on the outer one because the row arrays does not conform to Identifiable. I also changed my code to write out the views explicitly without the ForEach blocks but I get the same result: The column changes animate as movement but the row changes animate as fades.

How do I get it to animate all four views moving clockwise?

import SwiftUI

struct Block: Identifiable {
  var id : Int = 0
}

struct Board {
  let numRows = 2
  let numCols = 2
  
  var blocks: [[Block]]
  
  init() {
    blocks = Array(repeating: Array(repeating: Block(), count: numCols), count: numRows)
    
    for row in 0..<numRows {
      for col in 0..<numCols {
        blocks[row][col].id = row * numCols + col
      }
    }
  }
  
  mutating func rotate() {
    let temp = blocks[0][0]
    blocks[0][0] = blocks[1][0]
    blocks[1][0] = blocks[1][1]
    blocks[1][1] = blocks[0][1]
    blocks[0][1] = temp
    
  }
}

struct BlockView: View {
  var block: Block
  
  var body: some View {
    ZStack {
      Rectangle()
        .fill(Color.secondary)
      Text("\(block.id)")
        .font(.largeTitle)
    }
    .aspectRatio(1.0, contentMode: .fit)
  }
}

struct ContentView: View {
  @State var board = Board()
  
  var body: some View {
    VStack {
      Button("Rotate") {
        withAnimation(.easeInOut(duration: 1.0)) {
          board.rotate()
        }
      }
      
      VStack {
        ForEach(0..<2) { row in
          HStack {
            ForEach(board.blocks[row]) { block in
              BlockView(block: block)
            }
          }
        }
      }
    }
    .padding()
  }
}

#Preview {
  ContentView()
}```