When is DispatchIO cleanup handler called?

Question

I am creating DispatchIO channels using the file descriptor constructor:

self.channel = DispatchIO(type: .random, fileDescriptor: self.file, queue: fileProcessingQueue, cleanupHandler: { (errorCode) in
       // clean up when the channel closes
})

The documentation for cleanupHandler states:

"The handler to execute once the channel is closed."

The better documentation (when you switch language to ObjC) states:

"The block to enqueue when the system relinquishes control of the channel’s file descriptor."

"... the system takes control of the specified file descriptor until one of the following occurs:

You close the channel by calling the

dispatch_io_close

function.

..."

So from both of these, I expect the handler gets enqueued and executed if I call

self.channel.close(flags [.stop])

But it seems to never get executed under this condition. In fact, it's very difficult to get the cleanup handler to be called at all. The only way so far I have been able to get it to execute is by calling close() on the underlying file descriptor. Am I doing something wrong here? There's cleanup that I want to do when the channel closes that is different than the cleanup I want to do when I'm done with the file descriptor. Is this not the correct place to do it?

Concurrency

902

Posted by

fractal

Reply

Add a Comment

Answer 1

I’m not entirely sure what’s going in your situation but my experience is that

DispatchIO

works pretty well. Pasted is below is an example that simply streams through a file.

I’ll second john daniel’s recommendation that you read the doc comments in the headers. You should also avail yourself of the various man pages, starting with the

dispatch_io_create

man page.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

import Foundation

func main() {
    let io = DispatchIO(type: .stream, path: "/Users/quinn/Yo Ho Ho.dmg", oflag: O_RDONLY, mode: 0o666, queue: .main, cleanupHandler: { err in
        print("clean up, err: \(err)")
    })!
    func startRead() {
        io.read(offset: 0, length: 65536, queue: .main) { (done, dataQ, err) in
            let data = dataQ ?? DispatchData.empty
            if data.count != 0 {
                print("read, count: \(data.count)")
            } else {
                print("EOF")
                io.close(flags: [])
                return
            }
            if err != 0 {
                print("failed, err: \(err)")
                io.close(flags: [.stop])
                return
            }
            if done {
                startRead()
            }
        }
    }
    startRead()
    dispatchMain()
}

main()
exit(EXIT_SUCCESS)

Posted by

eskimo

Add a Comment

Answer 2

Am I doing something wrong here?

I can't answer that. All I can do is relate my own experiences with this API. To make a long story short, I found that there were so many different possible flows of execution that I felt that I had no real understanding of how to use the API. The documentation is very light and contradictory. For example, you found that you could only get the cleanup handler to execute if you called close() on the underlying file description. But one thing the documentation explicity says is "it is an error for your application to modify the file descriptor directly."

One thing I suggest is to look at the headers. There is often much better documentation there.

My dispatch IO code kept spiraling more and more out of control, getting more and more complicated. My ultimate decision was to abandon dispatch IO completely. Instead, I use old-school select(). It is conceptually a very small and simple API. I'm confident that I am able to handle all potential flows of execution. I have complete control over the data stream and the underlying file description. No, it's not pretty. No, it's not fancy. Maybe it is not very efficient. But it works reliably.

Posted by

john daniel

Add a Comment

Answer 3

Regarding programming errors, the documentation specifically states (emphasis mine):

"The channel takes control of the specified file descriptor until the channel closes ... It is a programmer error for you to modify the file descriptor while the channel owns it."

So this sounds like after I call channel.close() it is ok for me to close the underlying file descriptor. This is the scneario in which my cleanup handler is called. I close the channel, then I close the file descriptor, then my cleanup handler is enqueued. If I don't close the FD, I get no cleanup.

Your point about DispatchIO code spiraling out of control is well noted. I'm not too far along on this part of the project, but so far everything has been working as I expected, except for the cleanup handlers, however if weird behavior like this keeps cropping up I'll be prepared to go with a different solution. The key feature that drove me to DispatchIO in the first place is large (100's of MB) asynchronous, cancellable, streaming read operations. I'm definitely open to other suggestions.

Posted by

fractal

Add a Comment

Answer 4

So why not just manually call your cleanup code when you close the channel? Then let the cleanup handle anything that is remaining for the file descriptor.

I got confused not when it was working, but when it came time to deal with errors.

Posted by

john daniel

Add a Comment

Answer 5

Well, my actual situation is more complicated. I'm creating mutiple channels from the same FD, and each channel has some channel-specific cleanup code that needs to run when it's finished. The final cleanup of the FD happens much later. This is not an insurmountable problem, obviously I could manually call the channel-specific cleanup code when I close the channel, however it would be a lot more convenient and better factored if it was encapsualted in the cleanupHandler and worked like the documentation says.

Posted by

fractal

Add a Comment

Answer 6

I’m not entirely sure what’s going in your situation but my experience is that

DispatchIO

works pretty well. Pasted is below is an example that simply streams through a file.

I’ll second john daniel’s recommendation that you read the doc comments in the headers. You should also avail yourself of the various man pages, starting with the

dispatch_io_create

man page.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

import Foundation

func main() {
    let io = DispatchIO(type: .stream, path: "/Users/quinn/Yo Ho Ho.dmg", oflag: O_RDONLY, mode: 0o666, queue: .main, cleanupHandler: { err in
        print("clean up, err: \(err)")
    })!
    func startRead() {
        io.read(offset: 0, length: 65536, queue: .main) { (done, dataQ, err) in
            let data = dataQ ?? DispatchData.empty
            if data.count != 0 {
                print("read, count: \(data.count)")
            } else {
                print("EOF")
                io.close(flags: [])
                return
            }
            if err != 0 {
                print("failed, err: \(err)")
                io.close(flags: [.stop])
                return
            }
            if done {
                startRead()
            }
        }
    }
    startRead()
    dispatchMain()
}

main()
exit(EXIT_SUCCESS)

Posted by

eskimo

Add a Comment

Answer 7

Thanks! I was not aware dispatch had man pages. That explained exactly what I was seeing with the completion handlers. I can't use them the way I wanted, but at least I know what's going on now.

Posted by

fractal

Add a Comment

Answer 8

I'm creating mutiple channels from the same FD

In situations like this I often resort to having each channel

dup

the file descriptor. That way each channel can do its own clean up without coordinating with the others.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Posted by

eskimo

Add a Comment

When is DispatchIO cleanup handler called?

Accepted Reply

Replies