Is swiftData just not useable with large data?

Is SwiftData not designed for large datasets? How to create many instances of my Data Model?

I have a loop where I iterate over a CSV file of 30k+ lines. Totalling around 230k elements. I intend to save this data into SwiftData. However it takes around 90 seconds. In previous versions of my app I was using coreData. Then it took just 3 seconds. So after much investigation I have concluded the delay comes from creating instances of the Data Model. To Test this I have :

  • Removed any saves into SwiftData. So only creation of the Model Object occurs.

  • Created an exact copy of the SwiftData Model class but admitted the @Model macro

  • set isAutosaveEnabled to false

When I run my loop with the copy class it takes 3 seconds. So It must be creating the SwiftData model instances that causes the *30 delay.

The code using swiftData @Model. This loop takes 90+ seconds to complete

    class TimetableData {
     
        var arrivalTime : Int = 0
        var departureTime: Int = 0
        var departureRoute : Int = 0
        var directionOfTravel : Int = 0
        var dutyNumber : Int = 0
        var facilityIdentifier : String = ""
        var locationType : String = ""
        var mode : String = ""
        var nonStopStatus : Int = 0
        var recordIdentifier : String = ""
        var timetableIdentifier : Int = 0
        var trainNumber : String = ""
        var tripNumber : Int = 0
        var tripStartSite : String = ""
        var uniqueLocationCode : String = ""
        
        init(arrivalTime: Int = 0, departureTime: Int = 0, departureRoute: Int = 0, directionOfTravel: Int = 0, dutyNumber: Int = 0, facilityIdentifier: String = "", locationType: String = "", mode: String = "", nonStopStatus: Int = 0, recordIdentifier: String = "", timetableIdentifier: Int = 0, trainNumber: String = "", tripNumber: Int = 0, tripStartSite: String = "", uniqueLocationCode: String = "") {
        
            self.name = name
            self.arrivalTime = arrivalTime
            self.departureTime = departureTime
            self.departureRoute = departureRoute
            self.directionOfTravel = directionOfTravel
            self.dutyNumber = dutyNumber
            self.facilityIdentifier = facilityIdentifier
            self.locationType = locationType
            self.mode = mode
            self.nonStopStatus = nonStopStatus
            self.recordIdentifier = recordIdentifier
            self.timetableIdentifier = timetableIdentifier
            self.trainNumber = trainNumber
            self.tripNumber = tripNumber
            self.tripStartSite = tripStartSite
            self.uniqueLocationCode = uniqueLocationCode
        }
        
    }


for line in CSVFile {  

var columnValue = line.components(separatedBy: ",")

let timeTableDataEntry  = TimetableData (
                     
                            arrivalTime: Int(columnValue[0])!,
                            departureTime: Int(columnValue[1])!,
                            departureRoute: Int(columnValue[2])!,
                            directionOfTravel: Int(columnValue[3])!,
                            dutyNumber: Int(columnValue[4])!,
                            facilityIdentifier: columnValue[5],
                            locationType: columnValue[6],
                            mode: columnValue[7],
                            nonStopStatus: Int(columnValue[8])!,
                            recordIdentifier:  columnValue[9],
                            timetableIdentifier: Int(columnValue[10])!,
                            trainNumber: columnValue[11],
                            tripNumber: Int(columnValue[12])!,
                            tripStartSite: columnValue[13],
                            uniqueLocationCode:columnValue[14]
                        )

}

Then The exact same code but omitting the @Model takes around just 3 second.

   class TimetableDataForTesting {

        var arrivalTime : Int = 0
        var departureTime: Int = 0
        var departureRoute : Int = 0
        var directionOfTravel : Int = 0
        var dutyNumber : Int = 0
        var facilityIdentifier : String = ""
        var locationType : String = ""
        var mode : String = ""
        var nonStopStatus : Int = 0
        var recordIdentifier : String = ""
        var timetableIdentifier : Int = 0
        var trainNumber : String = ""
        var tripNumber : Int = 0
        var tripStartSite : String = ""
        var uniqueLocationCode : String = ""

        init(arrivalTime: Int = 0, departureTime: Int = 0, departureRoute: Int = 0, directionOfTravel: Int = 0, dutyNumber: Int = 0, facilityIdentifier: String = "", locationType: String = "", mode: String = "", nonStopStatus: Int = 0, recordIdentifier: String = "", timetableIdentifier: Int = 0, trainNumber: String = "", tripNumber: Int = 0, tripStartSite: String = "", uniqueLocationCode: String = "") {

            self.name = name
            self.arrivalTime = arrivalTime
            self.departureTime = departureTime
            self.departureRoute = departureRoute
            self.directionOfTravel = directionOfTravel
            self.dutyNumber = dutyNumber
            self.facilityIdentifier = facilityIdentifier
            self.locationType = locationType
            self.mode = mode
            self.nonStopStatus = nonStopStatus
            self.recordIdentifier = recordIdentifier
            self.timetableIdentifier = timetableIdentifier
            self.trainNumber = trainNumber
            self.tripNumber = tripNumber
            self.tripStartSite = tripStartSite
            self.uniqueLocationCode = uniqueLocationCode
        }

    }


for line in CSVFile {  

var columnValue = line.components(separatedBy: ",")

let timeTableDataEntry  = TimetableDataForTesting (

                            arrivalTime: Int(columnValue[0])!,
                            departureTime: Int(columnValue[1])!,
                            departureRoute: Int(columnValue[2])!,
                            directionOfTravel: Int(columnValue[3])!,
                            dutyNumber: Int(columnValue[4])!,
                            facilityIdentifier: columnValue[5],
                            locationType: columnValue[6],
                            mode: columnValue[7],
                            nonStopStatus: Int(columnValue[8])!,
                            recordIdentifier:  columnValue[9],
                            timetableIdentifier: Int(columnValue[10])!,
                            trainNumber: columnValue[11],
                            tripNumber: Int(columnValue[12])!,
                            tripStartSite: columnValue[13],
                            uniqueLocationCode:columnValue[14]
                        )

}

It does seem to be utterly useless for anything more than toy amounts of data.

Remembering that CoreData, is, utterly useless for anything more than toy amounts of data. this makes sense I guess.

@RyanTCB did you gather any more experience on this one?

Unfortunately no. Still very much waiting for solution. I have attempted running the code to save the data in a background thread. This does free up the UI but it then makes actually writing the data to the store take a very very long time. I would get 3 rows saved into database every second. but with 30k rows but at that rate it would take over 8 hours. Has to be a way. Can't think Apple would release something that's sooo bad especially when it should be built on top of something already tried and tested like Core Data.

Is swiftData just not useable with large data?
 
 
Q