Is SwiftData not designed for large datasets? How to create many instances of my Data Model?
I have a loop where I iterate over a CSV file of 30k+ lines. Totalling around 230k elements. I intend to save this data into SwiftData. However it takes around 90 seconds. In previous versions of my app I was using coreData. Then it took just 3 seconds. So after much investigation I have concluded the delay comes from creating instances of the Data Model. To Test this I have :
Removed any saves into SwiftData. So only creation of the Model Object occurs.
Created an exact copy of the SwiftData Model class but admitted the @Model macro
set isAutosaveEnabled to false
When I run my loop with the copy class it takes 3 seconds. So It must be creating the SwiftData model instances that causes the *30 delay.
The code using swiftData @Model. This loop takes 90+ seconds to complete
class TimetableData {
var arrivalTime : Int = 0
var departureTime: Int = 0
var departureRoute : Int = 0
var directionOfTravel : Int = 0
var dutyNumber : Int = 0
var facilityIdentifier : String = ""
var locationType : String = ""
var mode : String = ""
var nonStopStatus : Int = 0
var recordIdentifier : String = ""
var timetableIdentifier : Int = 0
var trainNumber : String = ""
var tripNumber : Int = 0
var tripStartSite : String = ""
var uniqueLocationCode : String = ""
init(arrivalTime: Int = 0, departureTime: Int = 0, departureRoute: Int = 0, directionOfTravel: Int = 0, dutyNumber: Int = 0, facilityIdentifier: String = "", locationType: String = "", mode: String = "", nonStopStatus: Int = 0, recordIdentifier: String = "", timetableIdentifier: Int = 0, trainNumber: String = "", tripNumber: Int = 0, tripStartSite: String = "", uniqueLocationCode: String = "") {
self.name = name
self.arrivalTime = arrivalTime
self.departureTime = departureTime
self.departureRoute = departureRoute
self.directionOfTravel = directionOfTravel
self.dutyNumber = dutyNumber
self.facilityIdentifier = facilityIdentifier
self.locationType = locationType
self.mode = mode
self.nonStopStatus = nonStopStatus
self.recordIdentifier = recordIdentifier
self.timetableIdentifier = timetableIdentifier
self.trainNumber = trainNumber
self.tripNumber = tripNumber
self.tripStartSite = tripStartSite
self.uniqueLocationCode = uniqueLocationCode
}
}
for line in CSVFile {
var columnValue = line.components(separatedBy: ",")
let timeTableDataEntry = TimetableData (
arrivalTime: Int(columnValue[0])!,
departureTime: Int(columnValue[1])!,
departureRoute: Int(columnValue[2])!,
directionOfTravel: Int(columnValue[3])!,
dutyNumber: Int(columnValue[4])!,
facilityIdentifier: columnValue[5],
locationType: columnValue[6],
mode: columnValue[7],
nonStopStatus: Int(columnValue[8])!,
recordIdentifier: columnValue[9],
timetableIdentifier: Int(columnValue[10])!,
trainNumber: columnValue[11],
tripNumber: Int(columnValue[12])!,
tripStartSite: columnValue[13],
uniqueLocationCode:columnValue[14]
)
}
Then The exact same code but omitting the @Model takes around just 3 second.
class TimetableDataForTesting {
var arrivalTime : Int = 0
var departureTime: Int = 0
var departureRoute : Int = 0
var directionOfTravel : Int = 0
var dutyNumber : Int = 0
var facilityIdentifier : String = ""
var locationType : String = ""
var mode : String = ""
var nonStopStatus : Int = 0
var recordIdentifier : String = ""
var timetableIdentifier : Int = 0
var trainNumber : String = ""
var tripNumber : Int = 0
var tripStartSite : String = ""
var uniqueLocationCode : String = ""
init(arrivalTime: Int = 0, departureTime: Int = 0, departureRoute: Int = 0, directionOfTravel: Int = 0, dutyNumber: Int = 0, facilityIdentifier: String = "", locationType: String = "", mode: String = "", nonStopStatus: Int = 0, recordIdentifier: String = "", timetableIdentifier: Int = 0, trainNumber: String = "", tripNumber: Int = 0, tripStartSite: String = "", uniqueLocationCode: String = "") {
self.name = name
self.arrivalTime = arrivalTime
self.departureTime = departureTime
self.departureRoute = departureRoute
self.directionOfTravel = directionOfTravel
self.dutyNumber = dutyNumber
self.facilityIdentifier = facilityIdentifier
self.locationType = locationType
self.mode = mode
self.nonStopStatus = nonStopStatus
self.recordIdentifier = recordIdentifier
self.timetableIdentifier = timetableIdentifier
self.trainNumber = trainNumber
self.tripNumber = tripNumber
self.tripStartSite = tripStartSite
self.uniqueLocationCode = uniqueLocationCode
}
}
for line in CSVFile {
var columnValue = line.components(separatedBy: ",")
let timeTableDataEntry = TimetableDataForTesting (
arrivalTime: Int(columnValue[0])!,
departureTime: Int(columnValue[1])!,
departureRoute: Int(columnValue[2])!,
directionOfTravel: Int(columnValue[3])!,
dutyNumber: Int(columnValue[4])!,
facilityIdentifier: columnValue[5],
locationType: columnValue[6],
mode: columnValue[7],
nonStopStatus: Int(columnValue[8])!,
recordIdentifier: columnValue[9],
timetableIdentifier: Int(columnValue[10])!,
trainNumber: columnValue[11],
tripNumber: Int(columnValue[12])!,
tripStartSite: columnValue[13],
uniqueLocationCode:columnValue[14]
)
}
RyanTCB.