TL;DR: Why is a hierarcy of serial queues the recommended way for managing concurrency in a modern application?
Years later, the recommendations made in WWDC 2017-709 "Modernizing Grand Central Dispatch Usage" regarding the use of a hierarchy of serial queues to manage concurrency in an application remain unclear to me. Old posts on former Apple mailing lists, StackOverflow and Swift's Forums add to the confusion. Hopefully there's an opportunity for some clarity here.
(I'm writing in the context of a macOS application developer.)
In the WWDC video, to improve concurrency performance, it's recommended that one should split up their application into sub-systems and back each sub-system by a serial queue. It's then recommended that those sub-systems should target a single, "root" queue that is also a serial queue. The talk mentions that use of serial queues improved concurrency performance in many of Apple's own application.
But with laptop and desktops having so many cores, I'm struggling to reconcile how running everything through a single serial queue helps with concurrency. On the surface, it feels like you'd be seriously under-utilizing available cores.
For example, I have an application that has the following sub-systems:
- Export Service - Used for exporting images or videos.
- Rendering Service - Used for rendering thumbnail previews.
- Caching Service - Used for caching random data.
- Database Service - Used for reads and writes to the database.
With the exception of maybe the database service, I want each of the other services to run as many concurrent requests and reasonable for the given machine. So each of those services is backed by a concurrent queue with an appropriate quality-of-service level. On a multi-core system, I should be able to render multiple thumbnails at once so using a serial queue does not make any sense. Same goes for exporting files. An export of a small image should not have to wait for the export of a large video to finish in front of it. So a concurrent queue is used.
Along with using sub-systems, the WWDC recommends that all sub-systems target a single, root serial queue. This doesn't make too much sense to me either because that implies that there's no reason to use a concurrent queue anywhere in your tree of queues because its concurrency is negated by the serial queue it targets, or at least that's how I understand it.
So if I did back each service by a serial queue, then target a root serial queue, I'd be in the situation where a thumbnail request has to wait for an export request to complete, which is not what I would want at all.
(The WWDC talk also makes heavy use of DispatchSources, but those are serial in execution as well.)
For the example sub-systems above, I actually use a hierarchy of concurrent queues that all target a root concurrent queue. Each sub-system runs at a different quality of service to help manage execution priority. In some cases, I manually throttle the number of concurrent requests in a given sub-system based on the available cores, as that seems to help a lot with performance. (For example, generating thumbnails of RAW files where it's better for me to explicitly restrict that to a maximum limit rather than relying on GCD.)
As someone who builds a ton of concurrency into their apps, and as someone who felt that they had a reasonably good grasp on how to use GCD, I've never been able to understand why a hierarchy of serial queues is the recommended way for doing concurrency in a modern app. Hopefully someone can shed a bit more light on that for me.