I'm using core data as part of a disk scan to get a map and total sizes for each folder within that search path. The issue I'm seeing is very high memory usage as every entity remains in memory, and I've so far not found out if there's a way of reducing it
The data model is this
The search goes depth first, creating an entity for each folder and file, the size of each file is then added to all of it's parent and grandparent folders. But because of the parent child relationship every entity seems to remain in memory. I need to keep a reference to the parents of the current folder I'm searching so I can add to their size, and I think the children relationship of these then has references to everything underneath leading to hundreds of thousands of entities in memory during the search.
Is there a way to just persist the entities so they're not kept in memory even if that means fetching a parent folder entity to update the size and then releasing it each time. It may slow the scan a lot but that would be fine to get around the memory issue. Or possibly would I need a different model to get around this?
I'm currently using the children relationship to then navigate the data after the scan, but maybe there's a way I could have a searchable path string rather than a relationship to get around the issue?