I have submitted a feature request for public Core Spotlight API: FB7136032.
I don't think the siloing of Core Spotlight is part of Apple's privacy effort or that it actually aligns with this. The privacy effort is focussed on user consent. Once consent is given the data should be readily available. This allows for application integration, automation, platform extension and avoids duplicated effort. To me it seems we are actually looking at an incomplete implementation. API to search Core Spotlight is missing. API to access mail messages is missing.
For example: once access to photos is granted, photos can be accessed via the file system, via scripting, via PHPhotoLibrary, and via MLMediaLibrary frameworks. All sorts of things become (resp. remain) possible. All hinges on user consent.
The current siloing and move to Core Spotlight has two problems:
- Much information is no longer available as siloing proceeds faster than API evolution. E.g. there is no API to access not notes or email messages. This limits integration and automation opportunities. In some cases, third-party developers can resort to duplicating effort. E.g. by direct access to IMAP servers
- Where public API to silos exist (PHPhotoLibrary, Contacts, …) the API lack the unifying nature of Spotlight / NSMetadataQuery. A public API to Core Spotlight should solve that.
-- Full text of FB7136032
In macOS Catalina, mail messages stored in ~/Library/Mail are no longer indexed by Spotlight. Since messages are still found by Spotlight searches, I assume indexing has been moved to CoreSpotlight.
This continues a trend where Safari bookmarks, Safari history, Apple notes, etc. have been confined to private databases and made searchable in the Spotlight window by way of CoreSpotlight.
Third-party applications have also adopted CoreSpotlight. Many “shoebox” applications can now make data available to the Spotlight window in ways that could not be done using the Spotlight index. Mostly because the data is not readily available as individual files.
For the most part this change needs to be applauded as more data becomes available for searches.
It however also creates problems:
- more and more data is stored in proprietary monolithic databases
- scripts and applications that rely on core applications as “system services” will break. E.g. it was possible to use Spotlight to watch for incoming mail messages and react
- generally speaking applications are cut off from collaboration
Much of this could be fixed by opening up the API used by the Spotlight window to search the CoreSpotlight indexes. Currently the CSSearchQuery allows only for searching the current application’s data.
Please extend CSSearchQuery or provide other public API to search all application data indexed by CoreSpotlight. I.e. an API to search Apple Mail messages, notes, …
BTW, it may have been too early to move Apple Mail messages to CoreSpotlight. The technological need is not there (yet). Mail messages are still stored as individual .emlx files and could still be indexed. I assume CoreSpotlight addresses other needs of the Mail application. Breaking other applications that relied on Spotlight as “public API” to Mail messages could be avoided by having a transition period where Mail messages are indexed in both Spotlight and CoreSpotlight. Spotlight access to Mail messages could be deprecated and ultimately removed once public API through CoreSpotlight is released.