How to manage NSDocument package with dynamic dir structure?

Hello!


I am trying to create a document package which stores many files in it (This will be an Electronic Health Record tool with lots of JPEGs, PDFs, etc) in a rather comples directory structure. Are there any examples for how to handle adding and removing files in NSDocument/UIDocument using NSFileWrapper? For example:


- When a user adds a JPEG/PDF to the document should A) I copy the file into the package using standard file system calls, or B) read the source file as NSData, add it to my model?


- If (B) above, then what might be an ideal way to create a model for a dir tree? NSMutableDictionary by { (NSString *) relativePath : (NSData *) fileData }?


- When a user deletes a file from the document package, should I (C) delete the file using standard fs calls and invalidate the (NSFileWrapper*) for it, or (D) just invalidate the file wrapper and assume the file will be deleted when the document is saved?


- How does saving in place or not saving in place affect the above?


- Is readFromFileWrapper called when a file is added to the document package on another device?


Thank you!

Accepted Reply

>> When a user adds a JPEG/PDF to the document should A) I copy the file into the package using standard file system calls, or B) read the source file as NSData, add it to my model?


Well, you can't do A. You can't really put a file into the document package before save time, because you don't know where that will be, and it would a terrible mistake to modify the original package. What I would try first is to make a place in your data model for an "imported" or "placed" image/file, and just put a URL there. At save time, when you're writing the actual document package, copy the data from this URL into the document. If you need the file contents for display in your UI, you might choose to keep a CGImage or NSImage or CGPDFDocument object in memory too. But there are 2 other considerations:


1. If the user can modify, move or delete the imported file after adding it to the document, you might want to grab its data instead of keeping a URL reference. You can either turn it into NSData, or copy it to a temporary location. The correct behavior here depends on the semantics of your app. If the user expects that they're placing a reference, a URL would do. If they expect they're placing the data, a URL is probably not the right way.


2. If you expect to be able to use memory mapped reads for these files (which, roughly, requires they're on a local file system), you might choose to use memory-mapped NSData objects instead, and forget about the explicit URLs.


But do the most natural thing first, and worry about excessive file-data-copying as a performance issue later.


>> When a user deletes a file from the document package, should I (C) delete the file using standard fs calls and invalidate the (NSFileWrapper*) for it, or (D) just invalidate the file wrapper and assume the file will be deleted when the document is saved?


Again, you can't delete the file, because you shouldn't (and maybe don't have permission to) modify the document until save time. Just delete the file's wrapper, and NSDocument will just not include the file in the saved package. It does the right thing for free!


>> How does saving in place or not saving in place affect the above?


When I was checking the documentation for the previous post, I came across a statement that in-place is needed for the NSDocument file coordination to be enabled. Save in place is more efficient, and can use the system-wide document versioning mechanism that powers the Time-Machine-style "history of time" view of older versions. This versioning also saves only changed file fragments, so it's more space-efficient than keeping multiple versions, too.


When you save in place, File -> Revert to Saved, changes to File -> Revert to Version… .

Replies

The typical approach, which works pretty well, is to start with the file wrapper (hierarchy) that you get when opening the document.


— The information from the referenced files is used to create your data model. (With file wrappers, you can read files lazily, which avoids a lag spike from reading everything at opening time. However, you will need to design your data models so that it "knows" when part of the data is missing, and have code to read it on demand, which affects your UI design, since it implies that the user will have to wait sometimes.)


— After that, do everything in terms of the data model. That is, when adding an image or PDF, add it (a URL to the source, presumably) to the data model, not to the document. The document and file wrappers are not really part of the data model.


— However, when you change the model in a way that invalidates an existing data file in the document package (either deleting or changing), then delete the file wrapper for that one file from the wrapper hierarchy. The point here is to leave untouched the file wrappers for files that are unchanged. Note that you can defer figuring out the wrapper changes until save time, but it's easier to do at the time that the model change is made.


— When your document is saved (which is likely to first happen at an autosave, about 20-30 seconds after a change is made), compare your data model to the file wrapper hierarchy, and create new file wrappers for files that are missing. From the above strategy, they'll be missing because the data is new or changed. Wrappers for deleted files will already be gone.


— Give the updated wrapper hierarchy to the saving mechanism. For individual file wrappers that you left untouched (so the corresponding files are unaltered) the NSDocument saving mechanism is smart enough not to copy or recreate the files, but typically can hard link them "for free". Only the new and changed files will be written. This is the most efficient way to save.


>> what might be an ideal way to create a model for a dir tree? NSMutableDictionary … ?


My advice is: use a hierarchy of custom objects with parent/child links. Arrays and dictionaries look like a free lunch, but for any reasonably complex app they will disappoint you later.


>> How does saving in place or not saving in place affect the above?


Do saving in place. The documentation says it's necessary for the file coordination behavior to work.


>> Is readFromFileWrapper called when a file is added to the document package on another device?


Er, I think so. Normally you don't have to think about it.

Thank you for taking the time to outline the philosophy for this. After reading it I am still unclear on how to answer these questions:

- When a user adds a JPEG/PDF to the document should A) I copy the file into the package using standard file system calls, or B) read the source file as NSData, add it to my model?

- When a user deletes a file from the document package, should I (C) delete the file using standard fs calls and invalidate the (NSFileWrapper*) for it, or (D) just invalidate the file wrapper and assume the file will be deleted when the document is saved?

- How does saving in place or not saving in place affect the above?

>> When a user adds a JPEG/PDF to the document should A) I copy the file into the package using standard file system calls, or B) read the source file as NSData, add it to my model?


Well, you can't do A. You can't really put a file into the document package before save time, because you don't know where that will be, and it would a terrible mistake to modify the original package. What I would try first is to make a place in your data model for an "imported" or "placed" image/file, and just put a URL there. At save time, when you're writing the actual document package, copy the data from this URL into the document. If you need the file contents for display in your UI, you might choose to keep a CGImage or NSImage or CGPDFDocument object in memory too. But there are 2 other considerations:


1. If the user can modify, move or delete the imported file after adding it to the document, you might want to grab its data instead of keeping a URL reference. You can either turn it into NSData, or copy it to a temporary location. The correct behavior here depends on the semantics of your app. If the user expects that they're placing a reference, a URL would do. If they expect they're placing the data, a URL is probably not the right way.


2. If you expect to be able to use memory mapped reads for these files (which, roughly, requires they're on a local file system), you might choose to use memory-mapped NSData objects instead, and forget about the explicit URLs.


But do the most natural thing first, and worry about excessive file-data-copying as a performance issue later.


>> When a user deletes a file from the document package, should I (C) delete the file using standard fs calls and invalidate the (NSFileWrapper*) for it, or (D) just invalidate the file wrapper and assume the file will be deleted when the document is saved?


Again, you can't delete the file, because you shouldn't (and maybe don't have permission to) modify the document until save time. Just delete the file's wrapper, and NSDocument will just not include the file in the saved package. It does the right thing for free!


>> How does saving in place or not saving in place affect the above?


When I was checking the documentation for the previous post, I came across a statement that in-place is needed for the NSDocument file coordination to be enabled. Save in place is more efficient, and can use the system-wide document versioning mechanism that powers the Time-Machine-style "history of time" view of older versions. This versioning also saves only changed file fragments, so it's more space-efficient than keeping multiple versions, too.


When you save in place, File -> Revert to Saved, changes to File -> Revert to Version… .

Huge, that was an excellent and comprehensive response to a design pattern that is referred to in the docs but not stated, thank you. Let's see how it goes in the code...

OK, it looks like this all works on the code end as well. Thank you so much for your great replies. They really filled a hole I've been workign on for a while now.

I've been racking my brain for months on how I was going to implement my document package, since I need URL's for the short videos that users include. It never occured to me to store them in a temporary directory until the document is saved. Thanks!

Any pointers on handling undo? Would it be better to put the undone videos in the same temporary directory, or trash them, enabling redo to retrieve them by storing the resulting url from the trash function? please answer here: https://forums.developer.apple.com/message/350198#350198