My app would like to know when another app has made changes in its documents folder.
Is there anyway to do this other then doing a complete scan of the folder?
There may be thousands of files so doing a full scan every time the app was brought into the foreground would be painfull.
There may be thousands of files so doing a full scan every time the app was brought into the foreground would be painfull.
Hmmm, there are two separate concepts present in that sentence:
Foreground vs background
Notifications vs full scans
On the foreground vs background issue, be aware that your Documents directory can be changed while your app is in the foreground (for example, iTunes file sharing via
UIFileSharingEnabledOn the notifications vs full scans issue, the standard notification for this is a kqueue, best accessed via a dispatch source. This only gives you a notification that something has changed in the directory. It does not tell you what’s changed. You will have to read the entire directory and compare it to your internal state.
Having said that, if you’re only expecting thousands of files that shouldn’t be too onerous even on the slowest modern (iOS 9 and later) iOS device. You’ll probably want to do the scan on a secondary thread though.
You can find official Apple sample code for the kqueue approach in the DocInteraction sample code. I’m not aware of any official Apple sample code for the dispatch source approach but I’ve pasted in some (Swift 3) code below that shows how it works.
Be aware that this stuff won’t resume your app in background when there are changes. Rather, if your app is suspended it’ll stay suspended, and when it’s next resumed you’ll get the notification.
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
let myEmail = "eskimo" + "1" + "@apple.com"class DirectoryMonitor {
typealias Delegate = DirectoryMonitorDelegate
init(directory: URL, matching typeIdentifier: String, requestedResourceKeys: Set<URLResourceKey>) {
self.directory = directory
self.typeIdentifier = typeIdentifier
self.requestedResourceKeys = requestedResourceKeys
self.actualResourceKeys = [URLResourceKey](requestedResourceKeys.union([.typeIdentifierKey]))
self.contents = []
}
let typeIdentifier: String
let requestedResourceKeys: Set<URLResourceKey>
private let actualResourceKeys: [URLResourceKey]
let directory: URL
weak var delegate: Delegate? = nil
private(set) var contents: Set<URL>
fileprivate enum State {
case stopped
case started(dirSource: DispatchSourceFileSystemObject)
case debounce(dirSource: DispatchSourceFileSystemObject, timer: Timer)
}
private var state: State = .stopped
private static func source(for directory: URL) throws -> DispatchSourceFileSystemObject {
let dirFD = open(directory.path, O_EVTONLY)
guard dirFD >= 0 else {
let err = errno
throw NSError(domain: POSIXError.errorDomain, code: Int(err), userInfo: nil)
}
return DispatchSource.makeFileSystemObjectSource(
fileDescriptor: dirFD,
eventMask: [.write],
queue: DispatchQueue.main
)
}
func start() throws {
guard case .stopped = self.state else { fatalError() }
let dirSource = try DirectoryMonitor.source(for: self.directory)
dirSource.setEventHandler {
self.kqueueDidFire()
}
dirSource.resume()
// We don't support `stop()` so there's no cancellation handler.
// kqueue.source.setCancelHandler {
// _ = close(...)
// }
let nowTimer = Timer.scheduledTimer(withTimeInterval: 0.0, repeats: false) { _ in
self.debounceTimerDidFire()
}
self.state = .debounce(dirSource: dirSource, timer: nowTimer)
}
private func kqueueDidFire() {
switch self.state {
case .started(let dirSource):
let timer = Timer.scheduledTimer(withTimeInterval: 0.2, repeats: false) { _ in
self.debounceTimerDidFire()
}
self.state = .debounce(dirSource: dirSource, timer: timer)
case .debounce(_, let timer):
timer.fireDate = Date(timeIntervalSinceNow: 0.2)
// Stay in the `.debounce` state.
case .stopped:
// This can happen if the read source fired and enqueued a block on the
// main queue but, before the main queue got to service that block, someone
// called `stop()`. The correct response is to just do nothing.
break
}
}
static func contents(of directory: URL, matching typeIdentifier: String, including: [URLResourceKey]) -> Set<URL> {
guard let rawContents = try? FileManager.default.contentsOfDirectory(
at: directory,
includingPropertiesForKeys: including,
options: [.skipsHiddenFiles]
) else {
return []
}
let filteredContents = rawContents.filter { url in
guard let v = try? url.resourceValues(forKeys: [.typeIdentifierKey]),
let urlType = v.typeIdentifier else {
return false
}
return urlType == typeIdentifier
}
return Set(filteredContents)
}
private func debounceTimerDidFire() {
guard case .debounce(let dirSource, let timer) = self.state else { fatalError() }
timer.invalidate()
self.state = .started(dirSource: dirSource)
let newContents = DirectoryMonitor.contents(of: self.directory, matching: self.typeIdentifier, including: self.actualResourceKeys)
let itemsAdded = newContents.subtracting(self.contents)
let itemsRemoved = self.contents.subtracting(newContents)
self.contents = newContents
if !itemsAdded.isEmpty || !itemsRemoved.isEmpty {
self.delegate?.didChange(directoryMonitor: self, added: itemsAdded, removed: itemsRemoved)
}
}
func stop() {
if !self.state.isRunning { fatalError() }
// I don't need an implementation for this in the current project so
// I'm just leaving it out for the moment.
fatalError()
}
}
fileprivate extension DirectoryMonitor.State {
var isRunning: Bool {
switch self {
case .stopped: return false
case .started: return true
case .debounce: return true
}
}
}
protocol DirectoryMonitorDelegate : AnyObject {
func didChange(directoryMonitor: DirectoryMonitor, added: Set<URL>, removed: Set<URL>)
}Thanks for the detailed answer.
As for documents changing when the app is in the foreground, is there anything other than access via ITunes that can do this? ITunes can only access what is in the root of the Documents folder, it cannot descend into sub folders as far as I know so it is limited in what it can do. This is not actually relevant to my app though since I do not plan on making the folder available via iTune's file sharing.
Using kqueue only works to detect changes made while the app is running so I will still need to do a scan when the app starts anyway. it would also require opening a file descriptor for every file which is not practical when you are dealing with 100,000 + files.
I had been hoping there was some magic way of doing this. 🙂
As for documents changing when the app is in the foreground, is there anything other than access via iTunes that can do this?
The new iOS 11 document browser can show the user your Documents directory, although you have to opt in to that via
UISupportsDocumentBrowserit would also require opening a file descriptor for every file
No, for every directory.
which is not practical when you are dealing with 100,000 + files.
OK, you need to explain more about how your app works here. A typical iOS document based app:
Should support iTunes file sharing
Should use the iOS 11 document browser
Should opt in to
UISupportsDocumentBrowserIs not going to be wrangling 100,000 documents
Clearly you’re not building a document based app. What are you building?
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
let myEmail = "eskimo" + "1" + "@apple.com"About the app:
It is a sync&share app that provides end to end encryption. It is a cloud based app but it doesn't fit the pattern of other cloud based apps because everything including the metadata is encrypted. This means I cannot fetch a content list from the serve, instead considerable processing needs to be done on the client side to decrypt the data and make it available to the user and then encrypt any changes the user may make and send them to the server. While doing all of this it is also updating its local metadata with any changes made by the remote clients.
Our customers are very concerned about security and so enabling iTunes file sharing is not something they want. If we provided support for document browsing I would still need some way to allow them to limit what was browsable.
The way I would envision this working, for example, is that the user edits a file in our documents folder using MS Word and save the changes. At this point the file has changed but it cannot be synced until the app is brought into the foreground. Once the app is active it recognizes that something has changed in its documents folder and syncs the changes with its central server. This isn't ideal but since the app is not allowed to run in the background for any length of time there is no way around it. What we need is a new 'service' class of apps that are allowed to execute continuously in the background but I do not see that coming any time soon. So the user needs to be aware that the file will not by synced until the app is moved to the foreground.
Re: kqueue:
In order to detect changes to a file you need to have a file handle open on the file itself. Monitoring folder with kqueue will only report files/folders being added/deleted in the monitored folder.
We do testing where the app is managing 100,000 + files so we need to be able to handle this case.
The way I would envision this working, for example, is that the user edits a file in our documents folder using MS Word and save the changes.
This is where I’m confused. How are you exposing this file in your Documents directory to other apps?
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
let myEmail = "eskimo" + "1" + "@apple.com"Currently we do not expose the files in the Documents directory. The user has to select the file in our app and do an "Open In" to pass it to MS Word. After the user has editied the file they can pass it back using "Open In" again. It works but it isn't the way users are use to working with documents.
But with IOS 11 I would like to make use of the ability to share the Documents directory with other apps to make this work flow more natural.
But with IOS 11 I would like to make use of the ability to share the Documents directory with other apps to make this work flow more natural.
And how are you planning to do that? Ignoring the specific issue of detecting modifications, what’s your overall strategy for sharing your Documents directory?
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
let myEmail = "eskimo" + "1" + "@apple.com"I think maybe this thread is digressing.
We already have a desktop version of the app that does all of this, i.e. stores documents in the Documents directory and detects when a change has occurred and then processes it.
Up until now our IOS version did not need to support this because the documents could only be accessed via the app itself which simplified things.
The reason for this thread was to find out if IOS provided any way to monitor what other apps were doing in our documents folder, assuming we supported UISupportsDocumentBrowser. I think the answer to this is 'No' unless we want to use kqueue and have a file handle open to every file and directory in my Documents directory which in our case is not piratical.
So if we want to use UISupportsDocumentBrowser we will need to do the same file scanning we currently do for the desktop version.
Thanks for your help on this.