-
Re: Plist max size
eskimo Feb 8, 2017 2:47 PM (in response to DenisO)I just wander, is it right approach to use plists for storing big tables, of I should use another way?
No, definitely not. Property lists are designed as a convenience mechanism for dealing with (relatively) small data sets. While things will probably work with a megabyte property list, there are definitely better options.
Is this a read-only data set bundled in your app? Or something you need to read and write?
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"
-
Re: Plist max size
DenisO Feb 9, 2017 12:55 AM (in response to eskimo)Yes, there is a read-only data sets.
So, what you suggest to replace plists?
Btw, I have another one app, for iPad, with plist. There only one array plist used, with about 100 dictionaries, every dictionary contains about 20-30 records, some of them pretty big (1000+ symbols string). Is it too big as well?
-
Re: Plist max size
eskimo Feb 9, 2017 2:06 AM (in response to eskimo)Yes, there is a read-only data sets.
OK. That’s important because dealing with read/write data is a whole different kettle of fish.
So, what you suggest to replace plists?
That depends on a bunch of other factors. What does this data look like? How are you accessing it? And do you need simultaneous access to all of your tables? Or just one table at a time?
When dealing with large data sets the goạl is to avoid bringing all of the data in memory. There’s three basic strategies for this:
Streaming
Exploit the VM system
Implement an object load > use > unload lifecycle
If your access pattern is linear, it makes sense to stream through the data with file system APIs. The canonical example of this is movie playback.
If the data will fit in virtual memory but is too big for physical memory, you might compile the data to a format that’s can be used directly and then memory map that file. The VM system will then take care of loading data as you access it, and unloading data when the system needs to recover physical memory.
In more complex cases it might make sense to use a database-style design. Both SQLite and Core Data work well for this. These support random access and complex queries.
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"
-
Re: Plist max size
DenisO Feb 9, 2017 6:45 AM (in response to eskimo)Its tables with numbers in format 00.00 or 0.00000
200-300 rows
about 100 columns
It’s a reference table, to get correct value by column and row name
As an example, here is multiply table
\ 1 2 3
1 1 2 3
2 2 4 3
3 3 6 9So, basically, to calculate some values, I need to get reference value from one or two tables.
For example
x = (a+b+c)/(get value from table A for 2*3)-
Re: Plist max size
eskimo Feb 9, 2017 3:06 PM (in response to DenisO)Is the table rectangular? That is, does each row have the same number of columns? And vice versa?
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"
-
Re: Plist max size
DenisO Feb 9, 2017 3:32 PM (in response to eskimo)Yes, it is
Each row have the same number of columns, and vice versa, yes.
real one will be similar with this - https://sanctusaura.files.wordpress.com/2010/06/relative-humidity2.jpg
-
Re: Plist max size
DenisO Feb 10, 2017 11:50 AM (in response to eskimo)Replied yesterday, but it still on moderation because of link to picture with table example.
So yes, all tables is rectangular, and each row have the same number of columns, and vice versa, yes.
-
-
Re: Plist max size
john daniel Feb 9, 2017 3:26 PM (in response to DenisO)300 (rows) * 100 (columns) * 8 (size of double) = 240000 bytes = 234K.
I think most people take up more space than that for icons on buttons.
-
-
Re: Plist max size
eskimo Feb 10, 2017 3:47 PM (in response to DenisO)john daniel wrote:
I think most people take up more space than that for icons on buttons.
Right. But this will be considerably bigger if you put it in an XML plist.
@DenisO, For a rectangular array like this I’d probably put the data into a file in binary form and then memory map that. You can then access the values at random and the VM subsystem will take care of the rest.
Furthermore, I’d wrap this in a nice, Swift-friendly data structure. Something like this:
class DataMap { let rowCount = 16 let columnCount = 16 init?(file: URL) { guard let d = try? NSData(contentsOf: file, options: [.alwaysMapped]) else { return nil } guard d.length == (self.rowCount * self.columnCount * MemoryLayout<Double>.size) else { return nil } self.data = d let base = d.bytes.assumingMemoryBound(to: Double.self) self.buffer = UnsafeBufferPointer(start: base, count: d.length) } private let data: NSData private let buffer: UnsafeBufferPointer<Double> subscript (row: Int, column: Int) -> Double { precondition((0..<self.rowCount).contains(row)) precondition((0..<self.columnCount).contains(column)) return self.buffer[row * self.rowCount + column] } }
Note that I’ve hard-coded the row and column counts, which makes sense if all of these objects have the same dimensions. If not, those values would need to be parameters to the initialiser.
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"
-
Re: Plist max size
DenisO Feb 11, 2017 6:18 AM (in response to eskimo)Thanks!
Will try this out
But, will try Core Data first -
Re: Plist max size
john daniel Feb 11, 2017 9:42 AM (in response to eskimo)I was thinking of the space required in memory rather than on disk.
Other options, like a binary file, SQLite database, or Core Data will certainly work, but each carries its own set of trade offs. picking the best format depends on how the data will eventually be used and/or how often the original data needs to be updated.
-
-
-
-
-
-
Re: Plist max size
KMT Feb 9, 2017 3:45 PM (in response to DenisO)I've gotten away with some crazy large read-only plist data sources, but usually they represented one table each. After a while, my issue became more of maintenance than deployment or performance, and moving away from a simple plist would have righted that ship, I'm sure.
If you have legitimately large data sets, and the skills to apply more comprehensive methods, I'd go that route.
-
Re: Plist max size
DenisO Feb 9, 2017 12:56 AM (in response to KMT)What "data storage" you using now, instead plists?
-
Re: Plist max size
KMT Feb 9, 2017 7:50 AM (in response to DenisO)Look into Apple's Core Data - which is basically a (non-trivial) data model abstraction layer over your data storage - allows you to manage relationships and fetching rules between different data entities, over large and complex data sets.
https://developer.apple.com/reference/coredata
Behind the Core Data layer, your data storage could be sqlite db, or binary object.
-
Re: Plist max size
DenisO Feb 10, 2017 11:57 AM (in response to KMT)Will check it closely, thanks
So far, I found it not really convenient to populate data to DB (need to write some code) and no useful features to put pre-populated DBs in app (need to write a code to replace empty files with full one from binary at first app start)
Work with plist file much easier for me so far-
Re: Plist max size
KMT Feb 10, 2017 1:55 PM (in response to DenisO)>Work with plist file much easier for me so far
I think it is common to start with a simple plist, then after you get the app in the store, and your skills and requirements grow, to migrate to CD, etc, later.
I think for now you'll be ok using a plist data source.
Good luck.
Ken
-
-
-
-
-
Re: Plist max size
DenisO Feb 15, 2017 12:02 AM (in response to DenisO)Done.
It works now.
Some main problems had to solve with CoreData:
1) Use pre-populated "DB" files in app
2) Make it work with iOS 8
3) Tranfser data from Excel to CSV, with table structure change (got 5000+ rows), and then, with separate app, to SQLite files
Now, need to polish and simplify fetch code
Thanks everyone!
You helped me a lot