Using URLProtocol with Caching

We want to create a custom URLProtocol that returns mock HTTP responses for testing.


This technique is explained in the following WWDC video (8:00)


https://developer.apple.com/videos/play/wwdc2018/417/


The WWDC example does not implement any of the URL Loading Systems caching API.


After implementing the caching APIs we've found that they do not work as expected.


Specifically, when using the default cache policy

.useProtocolCachePolicy
the loading system never passes a
cachedResponse
value to the protocol.


We expect the cached response to be passed to the protocol so it can inspect it and signal if it should be used. Otherwise we cannot see how a general URL Loading System, that provides caching, could integrate with custom protocols.


Example implementation:

public override init(request: URLRequest, cachedResponse: CachedURLResponse?, client: URLProtocolClient?) {
  super.init(request: request, cachedResponse: cachedResponse, client: client)

  // All cachedResponses are valid
  if let cachedResponse == cachedResponse {
    client?.urlProtocol(self, cachedResponseIsValid: cachedResponse)
  }
  
}

If the cached response is not passed, the custom protocol can only access the shared cache which would exclude any URLSessions that use a per session cache.


Have we misunderstood how the API is intended to be used, or is this a bug in the URL Loading System?

Replies

If the cached response is not passed, the custom protocol can only access the shared cache which would exclude any sessions that use a per session cache.

Indeed. This makes sense when you look at the history of this API.

NSURLProtocol
was introduced along with
NSURLConnection
, where there was no notion of a session. Thus, a protocol that wanted to use
NSURLCache
was expected to use the shared cache. When
NSURLSession
was introduced, we made no significant enhancements to
NSURLProtocol
, and thus there’s still no mechanism to get at the per-session cache.

If you read through the Caveats section of the CustomHTTPProtocol sample code’s read me, you’ll see that this shouldn’t come as a surprise. Even before the advent of

NSURLSession
,
NSURLProtocol
wasn’t up-to-date with respect to the latest
NSURLConnection
. I generally recommend that you steer clear of
NSURLProtocol
entirely. If I were in your shoes, I’d implement a shim layer within my networking abstraction and use that to insert my test infrastructure.

However, if you want to keep going down the

NSURLProtocol
path, one option here is to introduce a side channel to associate sessions with requests. For example, you could register your sessions with a centralised tracker, and then associate the tracking ID with the request via a custom URL property. Your protocol could then use that property to recover the session and thence any per-session properties.

You should also feel free to file a bug against

NSURLProtocol
requesting that it be updated for the modern world, although I have to set some expectations here. I filed a bunch of such bugs when I wrote CustomHTTPProtocol (again, see the read me) and none of them have been fixed )-:

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"