BACKGROUND
I am building an application for macOS. No existing users, so I can target arbitrary versions, but I'm not a paying developer yet, so no beta access.
This application works with an API provided by a remote server I do not control. This API provides a lot of objects with various types (for the purposes of this discussion, let's limit them to user and group). Each object is identified by a UUID. The type of the object is an attribute within the object, not some intrinsic aspect of how the object is structured. I can't tell what kind of object a given UUID represents ahead of time.
Groups have a 'members' attribute which is given to me as a list of UUIDs. Groups may contain other groups. The list of UUIDs comes in a stable order from fetch to fetch, but that order is not necessarily predictable (I think it depends on the order the user added the objects to the group). Further, when I get the list of all groups, they come in an order which is also stable, but also unpredictable.
I'm trying to import these objects into Core Data. The problem I'm running into is when groups reference each other. Sometimes, the inner group comes before the outer group in the API call result, and everything is fine. For other group pairs, sometimes the outer group comes first, so the inner group won't exist when I'm trying to build the reference. Effectively a race condition, even though the data is processed serially.
MY QUESTIONS
I'm thinking about running a fetch for every member UUID, creating a new entity if there is no result, running a fetch before creating any group, and updating the object if one already exists. That seems like it would be really expensive, though. In this particular situation, the number of groups within groups is always going to be far lower than the number of other things within groups, and I can fetch the user objects separately and commit them before the groups, thereby ensuring all users exist before groups try to reference them. Most data sets I expect to deal with have tens of thousands of users, and hundreds of groups with 10-2000 users each. Are fetch requests a lot cheaper than I expect?
I'm also thinking about blindly creating new entities for each member reference, then solving the duplicates when I commit (if one conflicting object only has a UUID and the other has a type, discard the one with no type). This seems like a much more robust solution, as the order of import would no longer matter for anything, allowing async import from the API. The problem is I haven't yet found details or sample code on how to make a custom NSMergePolicy. Is there sample code I have just missed?
I am building an application for macOS. No existing users, so I can target arbitrary versions, but I'm not a paying developer yet, so no beta access.
This application works with an API provided by a remote server I do not control. This API provides a lot of objects with various types (for the purposes of this discussion, let's limit them to user and group). Each object is identified by a UUID. The type of the object is an attribute within the object, not some intrinsic aspect of how the object is structured. I can't tell what kind of object a given UUID represents ahead of time.
Groups have a 'members' attribute which is given to me as a list of UUIDs. Groups may contain other groups. The list of UUIDs comes in a stable order from fetch to fetch, but that order is not necessarily predictable (I think it depends on the order the user added the objects to the group). Further, when I get the list of all groups, they come in an order which is also stable, but also unpredictable.
I'm trying to import these objects into Core Data. The problem I'm running into is when groups reference each other. Sometimes, the inner group comes before the outer group in the API call result, and everything is fine. For other group pairs, sometimes the outer group comes first, so the inner group won't exist when I'm trying to build the reference. Effectively a race condition, even though the data is processed serially.
MY QUESTIONS
I'm thinking about running a fetch for every member UUID, creating a new entity if there is no result, running a fetch before creating any group, and updating the object if one already exists. That seems like it would be really expensive, though. In this particular situation, the number of groups within groups is always going to be far lower than the number of other things within groups, and I can fetch the user objects separately and commit them before the groups, thereby ensuring all users exist before groups try to reference them. Most data sets I expect to deal with have tens of thousands of users, and hundreds of groups with 10-2000 users each. Are fetch requests a lot cheaper than I expect?
I'm also thinking about blindly creating new entities for each member reference, then solving the duplicates when I commit (if one conflicting object only has a UUID and the other has a type, discard the one with no type). This seems like a much more robust solution, as the order of import would no longer matter for anything, allowing async import from the API. The problem is I haven't yet found details or sample code on how to make a custom NSMergePolicy. Is there sample code I have just missed?
