How to setup Core Data with Core Spotlight without blocking main thread? (WWDC17-210)

Following along with WWDC17 Session 210 - What's new in Core Data:


I ticked a couple of string properties on an Entitiy in Core Data UI modeler - Index in Spotlight


then with the instantiated NSManagedObjectModel set an expression to the entity coreSpotlightDisplayNameExpression:

if let myEntity = mom.entitiesByName["MyEntity"] {
     myEntity.coreSpotlightDisplayNameExpression = NSExpression(block: { (object, expressions, dict) -> Any in
          let displayName = CSLocalizedString(localizedStrings: ["en": "My Item"])
          return displayName.localizedString()
     }
}

(Don't know if this NSExpression works at all as block not been called as of yet)


then subclassed NSCoreDataCoreSpotlightDelegate

class MyCoreDataCoreSpotlightDelegate: NSCoreDataCoreSpotlightDelegate {


     override func attributeSet(for object: NSManagedObject) -> CSSearchableItemAttributeSet? {
          guard let attributeSet = super.attributeSet(for: object) else { return nil }


          if "MyEntity" == object.entity.name {
               let item = object as! MyEntity
               attributeSet.displayName = item.itemName
               attributeSet.contentDescription = item.note
          }
          return attributeSet
     }
}

(Don't know if this override works at all as not been called as of yet)


Setting up Core Data with NSPersistentContainer:

let persistentStore = NSPersistentStoreDescription(url: dataStoreURL)
persistentStore.setValue("WAL" as NSObject?, forPragmaNamed: "journal_mode")
persistentStore.setOption(NSNumber(value: true), forKey:NSPersistentHistoryTrackingKey)


let coreSpotlightDelegate = MyCoreDataCoreSpotlightDelegate(forStoreWith:persistentStore, model: persistentContainer.managedObjectModel)
persistentStore.setOption(coreSpotlightDelegate, forKey:NSCoreDataCoreSpotlightExporter)

persistentContainer.persistentStoreDescriptions = [persistentStore]

persistentContainer.loadPersistentStores { storeDescription, error in

}


I have even added the Entitlement to talk to Core Spotlight 'com.apple.application-identifier = $(TeamIdentifierPrefix)$(CFBundleIdentifier)', just in case.....


Unfortunately, this setup results in the main thread being blocked as soon as loadPersistentStores is called. Cpu runs at about 30-40% continuously, disk activity 1 MB/s, memory doesnt seem to increase at all.


Pausing gives us a backtrace of:

* thread #1, queue = 'NSManagedObjectContext 0x6040003c7530', stop reason = signal SIGSTOP
  * frame #0: 0x0000000116fe47fe libsystem_kernel.dylib`semaphore_wait_trap + 10
    frame #1: 0x0000000116b598cf libdispatch.dylib`_dispatch_sema4_wait + 16
    frame #2: 0x0000000116b5a018 libdispatch.dylib`_dispatch_semaphore_wait_slow + 101
    frame #3: 0x00000001142d9aeb CoreData`-[NSCoreDataCoreSpotlightDelegate _updateSpotlightClientStateForHistoryTracking:] + 299
    frame #4: 0x00000001142d8ea7 CoreData`__73-[NSCoreDataCoreSpotlightDelegate _importObjectsUpdatedSinceTransaction:]_block_invoke + 263
    frame #5: 0x00000001140df748 CoreData`developerSubmittedBlockToNSManagedObjectContextPerform + 168
    frame #6: 0x0000000116b5933d libdispatch.dylib`_dispatch_client_callout + 8
    frame #7: 0x0000000116b60235 libdispatch.dylib`_dispatch_queue_barrier_sync_invoke_and_complete + 392
    frame #8: 0x00000001140df64e CoreData`-[NSManagedObjectContext performBlockAndWait:] + 286
    frame #9: 0x00000001142d8d3f CoreData`-[NSCoreDataCoreSpotlightDelegate _importObjectsUpdatedSinceTransaction:] + 287
    frame #10: 0x00000001142da05a CoreData`-[NSCoreDataCoreSpotlightDelegate _catchUpToCurrentTransaction] + 218
    frame #11: 0x000000011408a2d9 CoreData`-[NSPersistentStoreCoordinator addPersistentStoreWithType:configuration:URL:options:error:] + 969
    frame #12: 0x0000000114199f50 CoreData`-[NSPersistentStoreCoordinator _doAddPersistentStoreWithDescription:privateCopy:completionHandler:] + 336
    frame #13: 0x000000011419a361 CoreData`-[NSPersistentStoreCoordinator addPersistentStoreWithDescription:completionHandler:] + 177
    frame #14: 0x000000011414ee7c CoreData`-[NSPersistentContainer loadPersistentStoresWithCompletionHandler:] + 668
    frame #15: 0x000000010df48093 MyNewApp`SharedStoreHelper.init(storeName="0E4D1B65-58DB-464D-B64C-39D6C547F2F2", shareOwner="__defaultOwner__", coreSpotlightDelegateClass=MyNewApp.MyCoreDataCoreSpotlightDelegate, completion=0x000000010e06b360 MyNewApp`partial apply forwarder for closure #2 (Swift.Bool, MyNewApp.SharedStoreHelper) -> () in MyNewApp.ReloadStoresOperation.(reloadStores in _05B52F3B437462E7EAD7CB8BA8EFB1A6)(MyNewApp.SharedStoreHelper.Type, Swift.Optional<(Swift.Bool, Swift.Optional<MyNewApp.ArrayChanges>) -> ()>) -> () at ReloadStoresOperation.swift) at SharedStoreHelper.swift:101
    frame #16: 0x000000010e0b708d MyNewApp`MyNewAppSharedStoreDataManager.init(storeName="0E4D1B65-58DB-464D-B64C-39D6C547F2F2", shareOwner="__defaultOwner__", coreSpotlightDelegateClass=MyNewApp.MyCoreDataCoreSpotlightDelegate, completion=0x000000010e06b360 MyNewApp`partial apply forwarder for closure #2 (Swift.Bool, MyNewApp.SharedStoreHelper) -> () in MyNewApp.ReloadStoresOperation.(reloadStores in _05B52F3B437462E7EAD7CB8BA8EFB1A6)(MyNewApp.SharedStoreHelper.Type, Swift.Optional<(Swift.Bool, Swift.Optional<MyNewApp.ArrayChanges>) -> ()>) -> () at ReloadStoresOperation.swift) at MyNewAppSharedStoreDataManager.swift:0
    frame #17: 0x000000010e0b6df1 MyNewApp`MyNewAppSharedStoreDataManager.__allocating_init(storeName:shareOwner:coreSpotlightDelegateClass:completion:) at MyNewAppSharedStoreDataManager.swift:0
    frame #18: 0x000000010e06802b MyNewApp`ReloadStoresOperation.reloadStores(storeType=MyNewApp.MyNewAppSharedStoreDataManager, completionHandler=0x000000010defda10 MyNewApp`partial apply forwarder for closure #1 (Swift.Bool, Swift.Optional<MyNewApp.ArrayChanges>) -> () in closure #1 (__ObjC.NSPersistentStoreDescription, Swift.Optional<Swift.Error>) -> () in MyNewApp.MultiStoreManager.init(MyNewApp.SharedStoreHelper.Type, coreSpotlightDelegateClass: Swift.Optional<__ObjC.NSCoreDataCoreSpotlightDelegate.Type>, (Swift.Bool, MyNewApp.MultiStoreManager) -> ()) -> MyNewApp.MultiStoreManager at MultiStoreManager.swift, self=0x0000600000675140) at ReloadStoresOperation.swift:49
    frame #19: 0x000000010e0673f3 MyNewApp`ReloadStoresOperation.beginAsynchronousTask(self=0x0000600000675140) at ReloadStoresOperation.swift:24
    frame #20: 0x000000010e067424 MyNewApp`@objc ReloadStoresOperation.beginAsynchronousTask() at ReloadStoresOperation.swift:0
    frame #21: 0x0000000112a85ee8 Ensembles`-[CDEAsynchronousOperation start](self=0x0000600000675140, _cmd="start") at CDEAsynchronousOperation.m:50
    frame #22: 0x0000000112bb91e8 Foundation`__NSThreadPerformPerform + 334
    frame #23: 0x0000000114669101 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
    frame #24: 0x0000000114708f71 CoreFoundation`__CFRunLoopDoSource0 + 81
    frame #25: 0x000000011464da6b CoreFoundation`__CFRunLoopDoSources0 + 267
    frame #26: 0x000000011464cfff CoreFoundation`__CFRunLoopRun + 1279
    frame #27: 0x000000011464c889 CoreFoundation`CFRunLoopRunSpecific + 409
    frame #28: 0x00000001187539c6 GraphicsServices`GSEventRunModal + 62
    frame #29: 0x000000011060c5d6 UIKit`UIApplicationMain + 159
    frame #30: 0x000000010e07ee27 MyNewApp`main at AppDelegate.swift:18
    frame #31: 0x0000000116bd5d81 libdyld.dylib`start + 1


There is almost zero information about how to set this up (apple documentation or Stackoverflow etc.), other than the WWDC17 210 session. Does anyone have any ideas what to try next? or even if this is meant to work at all?


Many thanks in advance

Up vote post of DaveR Down vote post of DaveR
4.2k views
Posted by
DaveR
Reply
Add a Comment

Replies

I am seeing something similar when indexing. I thought she said in the video that the exporter was given a private context on a background thread but I'm not seeing it. If it's happening for you on initial load of the persisitent container, you could try setting:

persistentStore.shouldAddStoreAsynchronously = true

and seeing if that helps.


Saving any objects should trigger the attributeSet override, but I found that calling super.attributeSet(for:) always returned nil, so i just created a new one.


But you are right about a complete lack of information on how to properly set this up. I cannot find a way to delete indexed items from CoreSpotlight when i want to tear down the stack. Is the domainIdentifier set? I have no idea! I thought destroyPersistentStore might do this for free but it does not. I guess I have to manually delete every object before destroying the store which is not great.

Posted by
Procrastin8
Up vote reply of Procrastin8 Down vote reply of Procrastin8
Add a Comment

I followed your code closely, and I managed to get it work for newly added data AFTER I have added the above code. The delete works too, it dissapears in spotlight after a while.


I need it to index my existing stuffs, which it doesn't seems to do so! Based on your observation, it will? As you are complaining that it is doing this synchronously.

Posted by
ykphuah
Up vote reply of ykphuah Down vote reply of ykphuah
Add a Comment

Enabling `NSPersistentHistoryTrackingKey` seems to be the thing that causes everything to lock up.


Without that I find that it works for new data. If I perform an edit in my app then try to search for the edited item, spotlight has it.


What I can't figure out is how to convince core spotlight to index existing data. My app lets the user start with sample data, which behind the scenes is taken from a pre-built database, using `replacePersistentStore`.


None of the data from that operation appears to be indexed, and I can't find a way to either have it indexed when the building the sample database, or when replacing the user's database with it.


Core Spotlight seems to be rather badly documented on the Mac 😟

Posted by
elegantchaoscom
Up vote reply of elegantchaoscom Down vote reply of elegantchaoscom
Add a Comment

I know this thread is quite old but I wanted to call your attention to the fact the NSCoreDataCoreSpotlightDelegate object has been updated with new features and functionality! In our session, "Showcase App Data in Spotlight", you'll:

  • Discover how Core Data can surface data from your app in Spotlight with as little as two lines of code.
  • Learn how to make that data discoverable in Spotlight search and to customize how it is presented to people on device.
  • See how to implement full-text search within your app, driven completely with the data indexed by Spotlight.

We've released a [sample application](Showcase App Data in Spotlight) and updated our documentation to demonstrate how these new capabilities work.

For more information be sure to watch our session Showcase App Data in Spotlight on Wednesday.

Posted by
Frameworks Engineer
Post not yet marked as solved Up vote reply of Frameworks Engineer Down vote reply of Frameworks Engineer

  • Am I correct to assume that the new NSCoreDataCoreSpotlightDelegate stuff is for iOS only? I've tried using it on macOS but can't get it to work. For example, contrary to the documentation, Xcode claims that NSCoreDataCoreSpotlightDelegate has no method named startSpotlightIndexing.

    —
    Dalaplan
Add a Comment