Why does NSURLSession in iOS9 return -9802?

I had a related issue a while ago explained in this thread. Now, when trying to connect to s3.amazonaws.com, I'm getting a -9802 error. As per the other thread, I used TLSTool to help discover the security settings for the problematic URL, "https://s3.amazonaws.com/...":


$ TLSTool s_client -connect s3.amazonaws.com:443
*  input stream did open
* output stream did open
* output stream has space
* protocol: TLS 1.2
* cipher: ECDHE_RSA_WITH_AES_128_CBC_SHA
* trust result: unspecified
* certificate subjects:
*   0 s3.amazonaws.com
*   1 VeriSign Class 3 Secure Server CA - G3
*   2 VeriSign Class 3 Public Primary Certification Authority - G5
*  input stream has bytes
*  input stream end
* close
* bytes sent 0, bytes received 0


Note that the protocol is TLSv1.2 and that the cipher is in the approved list in the Apple Doc specifying the requirements.


Then, I tried playing around with exceptions settings in my plist, and found the magic values (actually, not sure I need the NSIncludesSubdomains):


<key>s3.amazonaws.com</key>
<dict>
<key>NSIncludesSubdomains</key>
<true/>
<key>NSExceptionRequiresForwardSecrecy</key>
<false/>
</dict>


What's up? Why do I need NSExceptionRequiresForwardSecrecy when I the site uses ECDHE_RSA_WITH_AES_128_CBC_SHA (on the list of approved forward ciphers) - bug? Am I missing some other red flag in the TLSTool output?


[I'm using the latest Xcode 7b4 ...]

Accepted Reply

What is not documented is that NSExceptionRequiresForwardSecrecy relaxes this check. The docs say that NSExceptionRequiresForwardSecrecy simply enables extra cypher suites and says nothing about relaxing the SHA-2/256 requirement. I've filed a bug (r. 22227634) to get the docs updated.

It turns out that the fact that

NSExceptionRequiresForwardSecrecy
relaxes the SHA-2/256 requirement is a bug; the intended behaviour of
NSExceptionRequiresForwardSecrecy
is the behaviour documented in the App Transport Security Technote, namely that it should just enable specific cypher suites.

Our plan is to fix this bug at some point in the future. We expect to fix this in some compatible fashion, so folks who’ve mistakenly used

NSExceptionRequiresForwardSecrecy
to disable the SHA-2/256 requirement won’t break. However, predicting the future is always a challenge.

Which brings us to what you should do right now. [A previous version of this post gave less concrete advice. The following is an update that tightens things up.] After discussing this with ATS Engineering our recommendations are:

  • If you're using a specific hosting service, you should consult your hosting service to get the latest advice.

  • In situations like this, where the server is fully compatible with ATS except for the SHA-2/256 certificate signing requirement, we recommend that you accurately document that state of affairs by using

    NSExceptionAllowsInsecureHTTPLoads
    .
  • You should attempt to make your server fully compatible with ATS as soon as possible.

  • When that happens, you should update your app with the more secure ATS settings.

I should stress that

NSExceptionAllowsInsecureHTTPLoads
isn't actually insecure. It's just as secure as your app currently is when it's running on iOS 8. Rather, it means that your app doesn't benefit from the extra security provided by ATS.

Share and Enjoy

Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Replies

[I'm using the latest Xcode 7b4 ...]

Please retry with Xcode 7.0b5. Stuff is changing pretty rapidly on this front and so I want to make sure we're all on the same page.

Share and Enjoy

Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Still not working in 7b5.


Hmmm - just read the Xcode 7b5 release notes. I see this:


Networking

Note

When negotiating a TLS/SSL connection with Diffie-Hellman key exchange, iOS 9 requires a 1024-bit group or larger. These connections include:

Secure Web (HTTPS)

...


Now, my forwared cipher is "ECDHE_RSA_WITH_AES_128_CBC_SHA" - hmmm - is DHE in that name stand for "Diffie-Hellman [key] Exchange"?


So the next question is - is the above a note on a limitation, or an announcement of what the future holds?


NOTE: there is this tidbit too - not sure if related or not:


Secure Transport

Note

DHE_RSA ciphersuites are now disabled by default in Secure Transport for TLS clients. This may cause failure to connect to TLS servers that only support DHE_RSA cipher suites. Applications that explicitly enable ciphersuites using

SSLSetEnabledCiphers()
are not affected and will still use DHE_RSA ciphersuites if explicetely enabled.

Still not working in 7b5.

Are you still seeing that

NSExceptionRequiresForwardSecrecy
works around the problem? I was playing around with this myself and that didn't work, so I'm curious about your experience.

Share and Enjoy

Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

dhoerl sent me the information I was looking for privately.

For those reading along at home, the reason why I was seeing weird results is that if you fetch the server's root (

https://s3.amazonaws.com
) via HTTPS, the server redirects you to an HTTP page. Once I disabled this redirect, I got a lot less confused.

I believe that the connection is failing the ATS check because the leaf certificate (not the TLS cypher suite, but the certificate itself) uses SHA-1 (sha1withRSAEncryption) whereas ATS requires SHA-2/256 or better (sha256WithRSAEncryption). This requirement is documented in the ATS Technote, although the discussion is not super clear.

What is not documented is that

NSExceptionRequiresForwardSecrecy
relaxes this check. The docs say that
NSExceptionRequiresForwardSecrecy
simply enables extra cypher suites and says nothing about relaxing the SHA-2/256 requirement. I've filed a bug to get the docs updated.

Anyway, the upshot of all of this is that dhoerl should set

NSExceptionRequiresForwardSecrecy
until such time as the server upgrades its certificate from SHA-1.

Share and Enjoy

Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

What is not documented is that NSExceptionRequiresForwardSecrecy relaxes this check. The docs say that NSExceptionRequiresForwardSecrecy simply enables extra cypher suites and says nothing about relaxing the SHA-2/256 requirement. I've filed a bug (r. 22227634) to get the docs updated.

It turns out that the fact that

NSExceptionRequiresForwardSecrecy
relaxes the SHA-2/256 requirement is a bug; the intended behaviour of
NSExceptionRequiresForwardSecrecy
is the behaviour documented in the App Transport Security Technote, namely that it should just enable specific cypher suites.

Our plan is to fix this bug at some point in the future. We expect to fix this in some compatible fashion, so folks who’ve mistakenly used

NSExceptionRequiresForwardSecrecy
to disable the SHA-2/256 requirement won’t break. However, predicting the future is always a challenge.

Which brings us to what you should do right now. [A previous version of this post gave less concrete advice. The following is an update that tightens things up.] After discussing this with ATS Engineering our recommendations are:

  • If you're using a specific hosting service, you should consult your hosting service to get the latest advice.

  • In situations like this, where the server is fully compatible with ATS except for the SHA-2/256 certificate signing requirement, we recommend that you accurately document that state of affairs by using

    NSExceptionAllowsInsecureHTTPLoads
    .
  • You should attempt to make your server fully compatible with ATS as soon as possible.

  • When that happens, you should update your app with the more secure ATS settings.

I should stress that

NSExceptionAllowsInsecureHTTPLoads
isn't actually insecure. It's just as secure as your app currently is when it's running on iOS 8. Rather, it means that your app doesn't benefit from the extra security provided by ATS.

Share and Enjoy

Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Well, I think I'll just use NSExceptionAllowsInsecureHTTPLoads. Really, this is Amazon - surely they'll get enough **** back to fix the issue in the not too distant future.


Thank you very much for all your help tracking down this gnarly issue!

Hi Dhoerl,


I was just wondering if the issues is ressolved on the beta 6? I still get the

NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9802)


And this is what is in my plist:

<key>NSAppTransportSecurity</key>
<dict>
      <key>NSExceptionDomains</key>
      <dict>
            <key>s3.amazonaws.com</key>
            <dict>
                <key>NSIncludeSubdomains</key>
                <true/>
                <key>NSThirdPartyExceptionAllowsInsecureHTTPLoads</key>
                <true/>
            </dict>
        </dict>
</dict>


I tried both NSThirdPartyExceptionAllowsInsecureHTTPLoads and NSExceptionAllowsInsecureHTTPLoads. Also tried the NSExceptionRequiresForwardSecrecy. Still no luck.


BUT


When I specify the full list mystuff.s3.amazonaws.com, it WORKS. Is it possible that NSIncludeSubdomains is broken?

Is it possible that NSIncludeSubdomains is broken?

AFAIK this works, although you do have to spell it correctly (-:

The documented value is

NSIncludesSubdomains
whereas you’re using
NSIncludeSubdomains
(you need an “s” at the end of “includes”).

Share and Enjoy

Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks a lot. That was it. Doh :/

Looks like an Amazon fix no later than September 30th.


https://aws.amazon.com/security/security-bulletins/aws-to-switch-to-sha256-hash-algorithm-for-ssl-certificates/

This is not the behavior we are seeing as of Xcode 7 GM and iOS 9 GM.


What we're seeing is that NSExceptionAllowsInsecureHTTPLoads = YES does not get around the SHA-256 requirement, but NSExceptionRequiresForwardSecrecy = NO does.

What we're seeing is that NSExceptionAllowsInsecureHTTPLoads = YES does not get around the SHA-256 requirement …

That makes no sense at all.

NSExceptionAllowsInsecureHTTPLoads
disables all ATS checking, so things behaviour like they did on iOS 8.

Moreover, I tested this again. First, I confirmed that the site dhoerl referenced is still using a SHA-1 signature.

$ TLSTool s_client -connect s3.amazonaws.com:443
*  input stream did open
* output stream did open
* output stream has space
* protocol: TLS 1.2
* cipher: ECDHE_RSA_WITH_AES_128_CBC_SHA
* trust result: unspecified
* certificate info:
*  0 rsaEncryption 2048 sha1-with-rsa-signature 's3.amazonaws.com'
…

Then I wrote a small test app that uses NSURLSession to fetch the root of that site (making sure to disable redirection, as I discussed on 11 Aug). With the following ATS settings it connected just fine.

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSExceptionDomains</key>
    <dict>
        <key>s3.amazonaws.com</key>
        <dict>
            <key>NSExceptionAllowsInsecureHTTPLoads</key>
            <true/>
        </dict>
    </dict>
</dict>

If you have a specific example of it failing, please post it.

Share and Enjoy

Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

https://www.dropbox.com/s/2oncfpyg9w2gy2j/TestAppTransportSecurity.zip?dl=0


Here's a sample project that has NSExceptionAllowsInsecureHTTPLoads.

Eskimo, I am seeing the same as Skunkworks. None of the per-domain settings seems to relax the SHA-256 setting except the buggy one.


Say you have a domain like s3.amazonaws.com, which supports TLSv1.2 and ECDHE, but the RSA 2048-bit cert is signed using sha1WithRSAEncryption. Without ATS exceptions, NSURLConnection fails to connect with kCFStreamErrorDomainSSL -9802 (errSSLFatalAlert).


But setting NSExceptionAllowsInsecureHTTPLoads=true or NSThirdPartyExceptionAllowsInsecureHTTPLoads=true for that domain doesn't fix it so that NSURLConnection connects. You still get -9802.


(THIS PARAGRAPH NOT FACT CHECKED THIS MORNING) Based on some previous experimentation that I haven't retried on the GM, it doesn't fail at the socket layer anymore, and you could implement the various NSURLAuthenticationChallenge related callbacks for NSURLAuthenticationMethodServerTrust to get around the default trust policy that disallows weak certificate fingerprints. But leave me implementing certificate trust myself or figuring out a proper incantation to SecTrustEvaluate. This seems frought with peril. (END OF DISCLAIMED SECTION)

However, if I set NSThirdPartyExceptionRequiresForwardSecrecy=false, suddenly everything just works. In fact, one of my service providers sent an email to their customers on iOS 9 release day telling us they hadn't managed to deploy their sha256 thumbprinted certs and actually suggested including this nonsense exception (nonsense because they do support ECDHE). This was my first inkling that it did not behave as specified in the technote.

Furthermore, it's super scary setting AllowsInsecureHTTPLoads, since the technote says "Use this key to access domains with no certificate, or with an error for a self-signed, expired, or hostname-mismatch certificate." And we do want HTTPS-only. And we don't want to turn on NSAllowsArbitraryLoads because we much prefer a whitelist of known good (or at least necessary) service providers with outdated TLS deployments.

You may want to retest on the public release. I am actually seeing different results between the GM and the final public release which I was not expecting.


The Twitter search api at https://api.twitter.com is similar to the Amazon S3 example above. It uses TLS 1.2 and ECDHE_RSA key exchange which provides forward secrecy but has a weakly signed SHA1 signature (see https://www.ssllabs.com/ssltest/analyze.html?d=api.twitter.com&s=199.59.148.87).


When testing with the GM I was seeing ATS rejections (error code -9802) which was fixed adding the NSThirdPartyExceptionAllowsInsecureHTTPLoads exception. This would seem to be expected if I have understood the ATS policy correctly.


What is odd is that with the public release of iOS 9 the exception is no longer required. The connection succeeds with the default ATS policy. Maybe the requirement for a SHA256 signed signature was relaxed but then at least the ATS Technote should be updated.


I have logged radar 22763438 with CFNetwork debug logs for both cases.