Implementing an iCloud on off button in SwiftUI results in crash

I've added iCloud into my SwiftUI app and everything seems to be working great, however I need to implement an on off toggle for it. After searching, I found a couple forums posts that suggested to re-create the container when icloud is toggled on off. Here's the code:

Code Block swift
lazy var persistentContainer: NSPersistentContainer = {
    return setupContainer()
}()
/* This is called when the iCloud setting is turned on and off */
func refreshCoreDataContainer() {
    /* Save changes before reloading */
    try? self.persistentContainer.viewContext.save()
    /* Reload the container */
    self.persistentContainer = self.setupContainer()
}
private func setupContainer() -> NSPersistentContainer {
    let useCloudSync = UserSettings.shared.enableiCloudSync
    let container: NSPersistentContainer!
    /* Use the icloud container if the user enables icloud, otherwise use the regular container */
    if useCloudSync {
        container = NSPersistentCloudKitContainer(name: "App")
    } else {
        container = NSPersistentContainer(name: "App")
        let description = container.persistentStoreDescriptions.first
        description?.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)
    }
    container.persistentStoreDescriptions.first?.setOption(true as NSNumber, forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey)
    /* Load the data */
    container.loadPersistentStores(completionHandler: { (storeDescription, error) in
        if let error = error as NSError? {
            fatalError("Unresolved error \(error), \(error.userInfo)")
        }
        container.viewContext.automaticallyMergesChangesFromParent = true
    })
    return container
}

The problem is that once I reload the container, the app crashes with this error:

Code Block 
Thread 1: "executeFetchRequest:error: A fetch request must have an entity."
Multiple NSEntityDescriptions claim the NSManagedObject subclass 'App.ColorCollection' so +entity is unable to disambiguate.
'ColorCollection' (0x60f000022000) from NSManagedObjectModel (0x607000067260) claims 'App.ColorCollection'.

I think the problem has to do with SwiftUI keeping a reference to the old container. When the window is created it is passing the container to it using the enviroment:

Code Block 
let contentView = MyContentView().environment(\.managedObjectContext, persistentContainer.viewContext)

So I tried to close the window, reload the container, then re create the window below, but the app still crashes.

Code Block 
func refreshCoreDataContainer() {
    print("reload")
    windowController.window?.close()
    /* Save changes before continuing */
    try? self.persistentContainer.viewContext.save()
    self.persistentContainer = self.setupContainer()
    self.createAndShowMainWindow()
}

How do I implement a iCloud toggle in SwiftUI without it crashing?