NSURLSession does not call delegate method for redirections from „http“ to „https"

I have encountered an issue with NSURLSession when a request to "http" results in a redirection to "https" with the same host and path (HTTP status code 302). In this case the delegate method "URLSession:task:willPerformHTTPRedirection:newRequest:completionHandler:" is not(!) called. For other redirections the delegate methods is called as expected.


In my App I'm using UIWebView to access web sites and NSURLProtocol to do filtering and other stuff, so I can see the details of the network traffic.


This iOS bug introduces some issues because the components which have initiated the request (like an instance of UIWebView) does not get notified about the redirection and assumes that the URL scheme is still "http". So even if the real URL of the site is a secure one, the web page still assumes an unsecure one. The Javascript property "location.href" contains the unsecure URL (with the scheme "http"). All relative URLs will be resolved with the wrong base URL etc.


Is there a worksround available for this? Am I doing something wrong? Why is NSURLSession forgetting to call the delegate method in this case?


Right now I'm checking in the delegate method "URLSession:dataTask:didReceiveResponse:completionHandler:" if the URL of the response has a different URL scheme ("https") than the initial request - if yes I call "[URLProtocolClient URLProtocol:wasRedirectedToRequest:request redirectResponse:]" in my NSURLProtocol class to "simulate" the missing redirection notification, so the caller (UIWebView) gets notified. But I'm not sure if this has some negative side effects, especially because I'm not yet sure if I've found all circumstances under which this bug within NSURLSession occurs and what exactly UIWebView is doing when it receives a redirection notification.

Replies

I’m not sure what’s going on in your specific case but this works for me. This delegate callback:

func urlSession(_ session: URLSession, task: URLSessionTask, willPerformHTTPRedirection response: HTTPURLResponse, newRequest request: URLRequest, completionHandler: @escaping (URLRequest?) -> Void) {
    NSLog("task session redirect from %@ to %@", response.url!.absoluteString, request.url!.absoluteString)
    completionHandler(request)
}

logs this redirect:

2017-02-21 21:18:03.670 QTestbed[21439:1719527] task session redirect from http://duckduckgo.com/ to https://duckduckgo.com/

when I create a data task for

<http://duckduckgo.com/>
.

I suspect that the problem here is your NSURLProtocol subclass. There’s some serious pitfalls getting redirects to work in that context. If you run the same test with the CustomHTTPProtocol sample code, does it exhibit the same problem.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I can definitely say that NSURLProtocol is not responsible for the issue. I've disabled my NSURLProtocol class and just called the NSURLSession API directly (just using the default configuration of NSURLSession). But the problem remains.


If I request "http://duckduckgo.com/" the server redirects to "https://duckduckgo.com/". After the request, the following delegate methods are called in this order:

  1. URLSession:task:didReceiveChallenge:completionHandler:
  2. URLSession:dataTask:didReceiveResponse:completionHandler:
  3. URLSession:task:didCompleteWithError:

So the delegate method for the redirection is NOT called.


If I request "http://apple.com/", the server redirects to "http://www.apple.com/". In this case the following delegate methods are called in this order:

  1. URLSession:task:willPerformHTTPRedirection:newRequest:completionHandler:
  2. URLSession:dataTask:didReceiveResponse:completionHandler:
  3. URLSession:task:didCompleteWithError:

In this case the delegate method for the redirection is called correctly.


If I request "https://apple.com/" (using a secure connection) , the server redirects to "https://www.apple.com/". In this case the following delegate methods are called in this order:

  1. URLSession:task:willPerformHTTPRedirection:newRequest:completionHandler:
  2. URLSession:task:didReceiveChallenge:completionHandler:
  3. URLSession:dataTask:didReceiveResponse:completionHandler:
  4. URLSession:task:didCompleteWithError:

In this case the delegate method for the redirection is also called correctly.


So only in those cases where the redirection changes the scheme, but not the domain part (another example where this happens would be "http://github.com/"), the delegate method for the redirection is NOT called. This all happens under iOS 10.2 (the latest public iOS release).

BTW: under iOS 9.3.x the delegate method for the redirection is always sent correctly. So this seems to be a bug of iOS 10.2 only (maybe other iOS 10 releases as well).

I haven't checked in 10.3 Beta yet, because this new beta and its Beta of XCode requires "macOS Sierra", which I do not have installed on my Mac for various reasons.

Well, that’s interesting. I tried again to reproduce the problem with DuckDuckGo without any success. Here’s what I did specifically:

  1. In Xcode 8.2, I created a new test project from the iOS > Single View Application template.

  2. I changed

    ViewController.m
    to the code pasted in below.
  3. In the

    Info.plist
    , I enabled
    NSAllowsArbitraryLoads
    .
  4. I ran it on a device running iOS 10.2.1.

Here’s what got logged:

2017-02-23 09:05:05.543511 xxoi[12702:7784093] redirect from http://duckduckgo.com/ to https://duckduckgo.com/
2017-02-23 09:05:05.650945 xxoi[12702:7784097] challenge NSURLAuthenticationMethodServerTrust
2017-02-23 09:05:05.742295 xxoi[12702:7784097] response
2017-02-23 09:05:05.743510 xxoi[12702:7784093] done

However, I then retried with

http://github.com/
and that does reproduce the problem:
2017-02-23 09:10:26.187070 xxoi[12724:7787629] challenge NSURLAuthenticationMethodServerTrust
2017-02-23 09:10:26.430530 xxoi[12724:7787633] response https://github.com/
2017-02-23 09:10:26.431890 xxoi[12724:7787632] done

Curiously, in that case I can then remove

NSAllowsArbitraryLoads
and it also works.

A quick check of the CFNetwork diagnostics log shows that the HTTP request never makes it to the ‘wire’. Somehow it’s getting rewritten to HTTPS before it’s seen by the core HTTP engine.

These two factoids (site-specific behaviour, never hitting the wire) got me thinking about HSTS. And some quick spelunking in the debugger reveals that this is, indeed, the culprit. GitHub has a built-in HSTS entry, so the HTTP request always gets rewritten as HTTPS. DuckDuckGo, OTOH, does not have a built-in HSTS entry, so the HTTP request hits the ‘wire’, triggers an HTTP redirect, which results in a

willPerformHTTPRedirection
delegate callback.

Finally, I suspect that you are seeing the problem with DuckDuckGo because your app has previously cached an HSTS requirement from that site. My app, OTOH, is brand new, so it’s relying on the built in HSTS requirements.

Interesting. It’s not obvious whether the HSTS rewrite should result in an

willPerformHTTPRedirection
delegate callback. I can see why you’d want it to, but it’d be a bit weird, architecturally, to synthesise an HTTP response in this case.

If you were the only client of these NSURLSession tasks then it would be easy to get around this problem by noticing the HTTPS URL in the response passed to the

didReceiveResponse
delegate callback. However, in your case UIWebView is the main client, and that complicates things.

I suspect the only way around this would be to have your NSURLProtocol subclass detect the HSTS rewrite and synthesise a redirect response to pass up to the web view.

Regardless of what else you do, you should file a bug about this oddity. In fact, it may make sense to file a suite of bugs, including:

  • A bug against the docs to explain this oddity

  • A feature request for how you’d like to see this work

Please post any bug numbers, just for the record.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"
#import "ViewController.h"

@interface ViewController () <NSURLSessionDataDelegate>

@property (nonatomic, strong, readwrite) NSURLSession * session;
@end

@implementation ViewController

- (void)viewDidLoad {
    [super viewDidLoad];
    NSURLSessionConfiguration * config = [NSURLSessionConfiguration defaultSessionConfiguration];
    self.session = [NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:nil];
    NSURL * url = [NSURL URLWithString:@"http://duckduckgo.com/"];
    [[self.session dataTaskWithURL:url] resume];
}

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task willPerformHTTPRedirection:(NSHTTPURLResponse *)response newRequest:(NSURLRequest *)request completionHandler:(void (^)(NSURLRequest *))completionHandler {
    NSLog(@"redirect from %@ to %@", response.URL, request.URL);
    completionHandler(request);
}

- (void)URLSession:(NSURLSession *)session dataTask:(NSURLSessionDataTask *)dataTask didReceiveResponse:(NSURLResponse *)response completionHandler:(void (^)(NSURLSessionResponseDisposition))completionHandler {
    NSLog(@"response");
    completionHandler(NSURLSessionResponseAllow);
}

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential * _Nullable))completionHandler {
    NSLog(@"challenge %@", challenge.protectionSpace.authenticationMethod);
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error {
    NSLog(@"done");
}

@end

Thank you very much for your response. Everthing makes perfect sense now.


For now, my workaround/solution looks like this:

In the "didReceiveResponse" method of my "NSURLProtocol" class, I check if the URL of the original request and the final destination are the same, besides the URL scheme, which has changed from "http" to "https". If this is the case and the "willPerformHTTPRedirection" method was not called for this request, I 'm calling [NSURLProtocolClient URLProtocol:wasRedirectedToRequest:redirectResponse:] to notify the component which made the request about the redirection. Of course I can not pass the "original" redirection response and requests here, because I do not have them, but I can create a "fake" one which includes at least all the values that are needed, borrow them from the response and request which I do have. And it seems to work fine so far. UIWebView seems to accept this and is working with the correct (secure) URLs now.


Bug ID: 30673726

I faced the kind of same issue. I'm also able to resolve it by using the "didReceiveResponse" method of my "NSURLProtocol" class.


When I received the call of the following method:

NSURLConnectionDataDelegate's -> (NSURLRequest *)connection:(NSURLConnection *)connection willSendRequest:(NSURLRequest *)request redirectResponse:(NSURLResponse *)redirectResponse in my NSURLProtocol, I keep the request and response for the redirect request in memory and pass that to [NSURLProtocolClient URLProtocol:wasRedirectedToRequest:redirectResponse:] to notify the component which made the request about the redirection. And it works fine.