NTLM ends up in a total lockup of all network traffic under iOS 10.1

I've encountered a very strange issue since iOS 10.1 in a custom App I wrote for a company. This App needs to access the internal network via VPN, and this internal company network uses NTLM athorization. The VPN connection is established by a third-party App (PulseSecure) and is working fine in general, so I think this is not the issue here.


The App uses UIWebView to access the internal company sites and also NSURLProtocol for certain important features which can not be implemented with WKWebView (like content filtering of the network traffic). The problem is not related to UIWebView, I can also reproduce it if I simply request all the resources of the site directly using NSURLSession. So for my test case I can simply use Apple's own "CustomHTTPProtocol" reference implementation for HTTP/HTTPS protocol handlers via NSURLProtocol and then send the HTTP request sequencially with a certain delay between all requests using the NSURLSession framework.


What happens is that if I send all the requests, with a delay of about 1-2 seconds between the requests, then after the 10th request, all network traffic shuts down for about 1 minute (some kind of timeout or so), then the network requests will continue to be processed and eventually shutting down after a while again.

The delay of 1-2 seconds between the request is more than enough to get the response from the server and finishing the requests successfully. So basically I send a request, get the response, wait the remaining time, send the next request, get the reponse etc. So at the time when the iOS stops processing network requests there are no unfinished network requests left. Which means for this 1 minute where the network layer is frozen, all the requests are queued up within NSURLSession but are not processed at all. After this minute they are processed again.


If I reduce the delay between the requests for example to only a small fraction of a second, then there will be multiple requests that needs to processed at the same time. But nevertheless the iOS will stop all network after processing the 10th request. Which means it does not matter how many requests are processed at the same time, and how many are waiting to be processed.


Only if I increase the delay between the requests to about 4-5 seconds, there's no more freezing network layer.


This problem seems to be related to NTLM authentication, all connections without NTLM do not cause this issue.


It looks like the problem is that the network layer only has a limited number of resources available for NTLM authorization which also seems to be busy for a while even after the request is already finished. And the network layer blocks if a certain number of NTLM authorization requests needs to be processed within a certain time (which seems to be about 4-5 seconds)


I do have found an ugly workaround for this iOS problem: Instead of using one single NSURLSession for all the network requests, I do create 15(!) sessions and distribute all requests to all these sessions equally. This way, when loading a web site (which can easily create about 90 requests), each session would only have to deal with a small number of the requests (in my example, each session only gets 6 requests), which is less than the 10 which have been found to be the limit which can be handled without a freezing network layer. And this does indeed avoid the freezing network traffic in my App.


While I can use this workaround for now, I wonder if there's a smarter workaround available for this problem? Are there any side effects when using so many NSURLSessions? Is there a better way to deal with this problem?



BTW: Bug ID #29383085

Replies

The App uses UIWebView to access the internal company sites and also NSURLProtocol for certain important features which can not be implemented with WKWebView (like content filtering of the network traffic).

I realise that this is orthogonal to this problem but the whole UIWebView + NSURLProtocol approach is less than ideal. For NTLM authentication it’s no longer necessary — WKWebView passes authentication challenges to its navigation delegate — but it seems like you’re also using it for other things (like content filtering). In that case, please make sure you file enhancement requests for anything missing from WKWebView.

BTW: Bug ID #29383085

Thanks for filing a bug about this. It would help if you could attach the following to your bug report:

While I can use this workaround for now, I wonder if there's a smarter workaround available for this problem?

It seems like that your problem is related to another problem I’ve seen (r. 28205623) that also involved mysterious stalls in NTLM-authenticated requests in iOS 10. The developer in that case was able to avoid the delays by tweaking the NSURLCredentialPersistence value they passed in while creating the credential. IIRC they previously had the parameter set to

.forSession
and changing it to
.none
avoiding the problem, but it may have been the other way around (-: Give that a go and let us know what you see.

Are there any side effects when using so many NSURLSessions?

NSURLSessions are meant to be long-lived objects, so we generally recommend that developers create a few sessions based on the type of requests they want to issue. So, this approach is not recommended in general. However, creating a bunch of sessions isn’t especially burdensome (especially for standard sessions; background sessions are a lot more heavyweight), so if this is enough to get your app up and limping then I’d say go for it.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks for your response.


You're right, I do need UIWebView and NSURLProtocol for content filtering and some other things where I need to be able to customize the HTTP requests. If I could use WKWebView, then NTLM would be not an issue (I've checked).


How does it work with the packet trace when I'm forced to use VPN to connect to this special internal network? Right now the company uses a special configuration profile on the device to make sure that the device is configured correctly. I guess the VPN tunnel will prevent that I can get a packet trace on my Mac.


I'm using standard sessions here, so I'll continue to use this workaround for now. The App also uses a background session, but only for downloads, which will only triggered by the user manually for specific files.So it is almost impossible that the user can start so many downloads manually to trigger this issue. Which mesn one background session should be fine.

How does it work with the packet trace when I'm forced to use VPN to connect to this special internal network?

That’s why I suggest a Remote Virtual Interface (RVI) packet trace; to quote QA1176

Important: The RVI represents the entire networking stack of the iOS device; you cannot target a specific interface on the device as you would on the Mac (using the

-i
option to
tcpdump
). However, information about the interface is recorded in the packet metadata. You can use
tcpdump
to display (via the
-k
option) and filter on (via the
-Q
option) this metadata. See the
tcpdump
online manual for details

In a VPN situation, RVI will see the traffic twice, one on the tunnel interface and one on the hardware interface being used by the tunnel interface.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"