Is Core Data the right tool?

My app generates piecemeal near terabyte data distributed over one hundred files. Due to RAM and internal HD limitations I must store to an external HD in piecemeal fashion. That is to say, append fresh records directly to an existing file without reading-in the whole file, appending, and writing it out again. Ultimately (when data generation ceases) I will check each file to ensure uniqueness of member data structures.

Is Core Data the tool I should use?

If not, how do I append directly to disk-files without first reading it in?

If I understand correctly your question, you can append data to a file directly:

https://stackoverflow.com/questions/27327067/append-text-or-data-to-text-file-in-swift

Thanks Claude31. Using the last entry in that link, I could make an array of FileHandleBuffer to handle the one hundred active files. However, it appears I must know the Data:Capacity:size ? Which I don't know.

Just answering your original question - Core data is not the right choice for this at all.

I agree with ABeeThatsBusy. You definitely don't want to attempt to do anything like this with Core Data.

I don't know what a FileHandleBuffer is. Are you talking about a FileHandle class? Don't ever use that class. It can throw Objective-C exceptions. If you are managing terabytes of data, those would be a regular occurrence.

The lowest level C file descriptors can append data to an existing file. Many of Apple's higher-level convenience methods and classes assume that you will have all data in memory and every write will completely replace the existing file. Don't do that.

Are you talking about a FileHandle class? Don't ever use that class. It can throw Objective-C exceptions.

The modern API for FileHandle no longer throws Objective-C exceptions. Contrast:

Having said that, I generally agree with your sentiment that a low-level API is likely to be the right choice for this task. When working in Swift I recommend Swift System. You can then build your own abstraction layered on top of that.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Is Core Data the right tool?
 
 
Q