How to find all files in subdirectories with NSMetadataQuery?

>>The following properly lists files in subdirectories on macOS and iOS when the search scope is an iCloud container, but returns nothing when run on an arbitrary folder on MacOS: >Does anyone know how to query for all files in a folder on the filesystem? The docs say to use "NSMetadataItemFSNameKey == *" but that throws an exception

Is your app sandboxed? If so, do you have proper permission to search at that location on the file system? Is your app macOS or iOS (or both?).

I haven't tried to use NSMetadataQuery recently but I remember being extremely frustrated the last time I tried to work with that API for iCloud on iOS. It's just a weird API IMO.

On macOS I don't think there is really any reason to use NSMetadataQuery for your document based app to detect changes inside your package though. You should be able to override the NSFilePresenter methods on your NSDocument subclass I think. Though I recall there being a couple longstanding bugs where NSFIlePresenter's subItem events didn't get called, or got called at the wrong time, or something. On macOS I was under the impression that NSDocument "just worked" with iCloud and you normally don't care if your doc is in iCloud or not. You just let it do what it do, (though there may be special circumstances where you'd need to use NSMetadataQuery). On iOS document based apps used to be a major pain to make, but supposedly that has gotten better.

Thank you for your reply. I don't know anything about NSFilePresenter yet and will have a look to see how it may interact with this functionality.

However, I should have been more clear about my question. The predicate...

@"%K LIKE '*'", NSMetadataItemFSNameKey

...works recursively on macOS and iOS with the proper iCloud entitlements. This code is complete and functional on those platforms. But for apps compiled without the iCloud entitlement on macOS, this predicate returns nothing and this modfied version works for the passed search path but not recursively...

@"%K LIKE '*.fd'", NSMetadataItemFSNameKey

This is the use case when I set a search path to my file package when compiling a pure C++/ObjC app from the command line without Xcode, and so without entitlements (for various reaons).

So I am trying to figure out how to recursively search an arbitrary path on the disk when not using Xcode and iCloud entitlements.

I don't know the direct answer to your question either. For this kind of issue, the best solution is often to submit a TSI (tech support incident, though I think it's called something different now) for your developer account, to get an Apple engineer to look at the problem.

However, I think Mr. Savage is correct in directing your attention away from NSMetadataQuery:

>> On macOS I don't think there is really any reason to use NSMetadataQuery for your document based app to detect changes inside your package though. You should be able to override the NSFilePresenter methods on your NSDocument subclass I think.

Using NSMetadataQuery seems like the wrong approach. For a start, the document might be on a volume or in a folder for which the user has disabled Spotlight, in which case the monitoring simply isn't going to work. If the monitoring app is also displaying the entire (in some sense) document, then coordinated access through NSFileCoordinator/NSFilePresenter seems like the correct approach, because it ensures the document is always presented in a consistent state.

Or, if your app (command line app, is how you described it) is simply waiting for changes to specific files in the document package individually, then monitoring the file system directly would be a more usual approach:

https://developer.apple.com/library/content/documentation/Darwin/Conceptual/FSEvents_ProgGuide/Introduction/Introduction.html

Maybe I misunderstand, but it seems to me that the app you're talking about is only going to deal with local document packages, and the packages are known explicitly. In that case, it does seem that you need file system monitoring, not Spotlight searching.

That is correct, thank you.

One problme though, now that I correctly set up the Export UTI entry so that macos sees the package as a file instead of a dir, the NSMetadataQuery isn't searching them anymore when run through Xcode and iCloud is enabled. Any thoughts there?

Hmm, not really. Do changes within the package produce a notification for the whole package?

I think you're really into TSI territory here, but it does occur to me that if you update the "interior" of the package due to a download, and make sure the package itself (at the top level) gets its modification date updated, you should at least get one notification, and you can can search within the package for relevant changes.

However, the idea of updating a document (i.e. NSDocument) package without proper file coordination is going to be dangerous if the document can be saved simultaneously in another (i.e. local editing) app.

You have a pretty subtle set of requirements here, and the solution isn't obvious.

Interesting. Maybe a TSI is the way to go. How is this normally done with document packages, assuming that the package is only edited via NSDocument calls (which appears to be an NSFilePresenter) by apps with the proper iCloud entitlements (forget the whole command line thing for a moment; that is a convenience use case which only pertains to a development phase of this PyQt app). Does one always treat the package as a single file on iCloud, where the download status of the root package path reflects the status of all the files within it, and then the user just has the option to download the whole package or not? Do you think that I am causing unnecessary grief for myself trying to add additional granularity of control on the download status within the package? Thanks, this is being very helpful!

It's a while since I used "ubiquitous" package documents, but my recollection is that they're treated as units, which is consistent with the behavior of NSMetadataQuery on packages. Yes, AFAIK the user has no control over the location of parts of the package. My guess is that iCloud doesn't even store the package as separate files, so there are no individual files to track.

I would suggest you do this the natural way (treating the file package as indivisible, except within your NSDocument subclass), and see how far you can satisfy your needs under that approach.

I think this is conceptually coming together, and I think you are right. I have read this mountanous web of iCloud/Document docs but I think your tips are finally making it make sense in my mind. I need to just make use of NSFileWrapper and stick to the wrapper validation/invalidation scheme outlined in the docs for Document-based apps. Hypothetically speaking, I should be able to detect the addition and deletion of new files within my package when revealed by the NSFileWrapper tree in readFromFileWrapper and fileWrapperOfType.

From the docs:

For more complex documents, you might define an arbitrarily deep tree of directories, files, and symbolic links, which you traverse by recursively querying the fileWrappers property of the directory file wrappers at each level.

(https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileWrappers/FileWrappers.html#//apple_ref/doc/uid/TP40010672-CH13-DontLinkElementID_2)

Document-based apps use file coordination automatically, and your app does not need to do anything special to adopt it.

(https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileWrappers/FileWrappers.html#//apple_ref/doc/uid/TP40010672-CH13-DontLinkElementID_8)