17 Replies
      Latest reply: Sep 22, 2015 12:44 PM by ifdev RSS
      dhoerl Level 1 Level 1 (0 points)

        I had a related issue a while ago explained in this thread. Now, when trying to connect to s3.amazonaws.com, I'm getting a -9802 error. As per the other thread, I used TLSTool to help discover the security settings for the problematic URL, "https://s3.amazonaws.com/...":

         

        $ TLSTool s_client -connect s3.amazonaws.com:443
        *  input stream did open
        * output stream did open
        * output stream has space
        * protocol: TLS 1.2
        * cipher: ECDHE_RSA_WITH_AES_128_CBC_SHA
        * trust result: unspecified
        * certificate subjects:
        *   0 s3.amazonaws.com
        *   1 VeriSign Class 3 Secure Server CA - G3
        *   2 VeriSign Class 3 Public Primary Certification Authority - G5
        *  input stream has bytes
        *  input stream end
        * close
        * bytes sent 0, bytes received 0
        
        

         

        Note that the protocol is TLSv1.2 and that the cipher is in the approved list in the Apple Doc specifying the requirements.

         

        Then, I tried playing around with exceptions settings in my plist, and found the magic values (actually, not sure I need the NSIncludesSubdomains):

         

        <key>s3.amazonaws.com</key>
        <dict>
        <key>NSIncludesSubdomains</key>
        <true/>
        <key>NSExceptionRequiresForwardSecrecy</key>
        <false/>
        </dict>
        
        

         

        What's up? Why do I need NSExceptionRequiresForwardSecrecy when I the site uses ECDHE_RSA_WITH_AES_128_CBC_SHA (on the list of approved forward ciphers) -  bug? Am I missing some other red flag in the TLSTool output?

         

        [I'm using the latest Xcode 7b4 ...]

        • Re: Why does NSURLSession in iOS9 return -9802?
          eskimo Apple Staff Apple Staff (6,505 points)

          [I'm using the latest Xcode 7b4 ...]

          Please retry with Xcode 7.0b5.  Stuff is changing pretty rapidly on this front and so I want to make sure we're all on the same page.

          Share and Enjoy

          Quinn "The Eskimo!"
          Apple Developer Relations, Developer Technical Support, Core OS/Hardware
          let myEmail = "eskimo" + "1" + "@apple.com"

            • Re: Why does NSURLSession in iOS9 return -9802?
              dhoerl Level 1 Level 1 (0 points)

              Still not working in 7b5.

               

              Hmmm - just read the Xcode 7b5 release notes. I see this:

               

              Networking

              Note

              When negotiating a TLS/SSL connection with Diffie-Hellman key exchange, iOS 9 requires a 1024-bit group or larger. These connections include:

              • Secure Web (HTTPS)

                 ...

               

              Now, my forwared cipher is "ECDHE_RSA_WITH_AES_128_CBC_SHA" - hmmm - is DHE in that name stand for "Diffie-Hellman [key] Exchange"?

               

              So the next question is - is the above a note on a limitation, or an announcement of what the future holds?

               

              NOTE: there is this tidbit too - not sure if related or not:

               

              Secure Transport

              Note

              • DHE_RSA ciphersuites are now disabled by default in Secure Transport for TLS clients. This may cause failure to connect to TLS servers that only support DHE_RSA cipher suites. Applications that explicitly enable ciphersuites using SSLSetEnabledCiphers() are not affected and will still use DHE_RSA ciphersuites if explicetely enabled.
                • Re: Why does NSURLSession in iOS9 return -9802?
                  eskimo Apple Staff Apple Staff (6,505 points)

                  Still not working in 7b5.

                  Are you still seeing that NSExceptionRequiresForwardSecrecy works around the problem?  I was playing around with this myself and that didn't work, so I'm curious about your experience.

                  Share and Enjoy

                  Quinn "The Eskimo!"
                  Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                  let myEmail = "eskimo" + "1" + "@apple.com"

                    • Re: Why does NSURLSession in iOS9 return -9802?
                      eskimo Apple Staff Apple Staff (6,505 points)

                      dhoerl sent me the information I was looking for privately.

                      For those reading along at home, the reason why I was seeing weird results is that if you fetch the server's root (https://s3.amazonaws.com) via HTTPS, the server redirects you to an HTTP page.  Once I disabled this redirect, I got a lot less confused.

                      I believe that the connection is failing the ATS check because the leaf certificate (not the TLS cypher suite, but the certificate itself) uses SHA-1 (sha1withRSAEncryption) whereas ATS requires SHA-2/256 or better (sha256WithRSAEncryption).  This requirement is documented in the ATS Technote, although the discussion is not super clear.

                      What is not documented is that NSExceptionRequiresForwardSecrecy relaxes this check.  The docs say that NSExceptionRequiresForwardSecrecy simply enables extra cypher suites and says nothing about relaxing the SHA-2/256 requirement.  I've filed a bug to get the docs updated.


                      Anyway, the upshot of all of this is that dhoerl should set NSExceptionRequiresForwardSecrecy until such time as the server upgrades its certificate from SHA-1.

                      Share and Enjoy

                      Quinn "The Eskimo!"
                      Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                      let myEmail = "eskimo" + "1" + "@apple.com"

                        • Re: Why does NSURLSession in iOS9 return -9802?
                          eskimo Apple Staff Apple Staff (6,505 points)

                          What is not documented is that NSExceptionRequiresForwardSecrecy relaxes this check.  The docs say that NSExceptionRequiresForwardSecrecy simply enables extra cypher suites and says nothing about relaxing the SHA-2/256 requirement.  I've filed a bug (r. 22227634) to get the docs updated.

                          It turns out that the fact that NSExceptionRequiresForwardSecrecy relaxes the SHA-2/256 requirement is a bug; the intended behaviour of NSExceptionRequiresForwardSecrecy is the behaviour documented in the App Transport Security Technote, namely that it should just enable specific cypher suites.

                          Our plan is to fix this bug at some point in the future.  We expect to fix this in some compatible fashion, so folks who’ve mistakenly used NSExceptionRequiresForwardSecrecy to disable the SHA-2/256 requirement won’t break.  However, predicting the future is always a challenge.

                          Which brings us to what you should do right now.  [A previous version of this post gave less concrete advice.  The following is an update that tightens things up.]  After discussing this with ATS Engineering our recommendations are:

                          • If you're using a specific hosting service, you should consult your hosting service to get the latest advice.

                          • In situations like this, where the server is fully compatible with ATS except for the SHA-2/256 certificate signing requirement, we recommend that you accurately document that state of affairs by using NSExceptionAllowsInsecureHTTPLoads.

                          • You should attempt to make your server fully compatible with ATS as soon as possible.

                          • When that happens, you should update your app with the more secure ATS settings.

                          I should stress that NSExceptionAllowsInsecureHTTPLoads isn't actually insecure.  It's just as secure as your app currently is when it's running on iOS 8.  Rather, it means that your app doesn't benefit from the extra security provided by ATS.

                          Share and Enjoy

                          Quinn "The Eskimo!"
                          Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                          let myEmail = "eskimo" + "1" + "@apple.com"

                            • Re: Why does NSURLSession in iOS9 return -9802?
                              dhoerl Level 1 Level 1 (0 points)

                              Well, I think I'll just use NSExceptionAllowsInsecureHTTPLoads. Really, this is Amazon - surely they'll get enough blow back to fix the issue in the not too distant future.

                               

                              Thank you very much for all your help tracking down this gnarly issue!

                              • Re: Why does NSURLSession in iOS9 return -9802?
                                skunkworks Level 1 Level 1 (0 points)

                                This is not the behavior we are seeing as of Xcode 7 GM and iOS 9 GM.

                                 

                                What we're seeing is that NSExceptionAllowsInsecureHTTPLoads = YES does not get around the SHA-256 requirement, but NSExceptionRequiresForwardSecrecy = NO does.

                                  • Re: Why does NSURLSession in iOS9 return -9802?
                                    eskimo Apple Staff Apple Staff (6,505 points)

                                    What we're seeing is that NSExceptionAllowsInsecureHTTPLoads = YES does not get around the SHA-256 requirement …

                                    That makes no sense at all.  NSExceptionAllowsInsecureHTTPLoads disables all ATS checking, so things behaviour like they did on iOS 8.

                                    Moreover, I tested this again.  First, I confirmed that the site dhoerl referenced is still using a SHA-1 signature.

                                    $ TLSTool s_client -connect s3.amazonaws.com:443
                                    *  input stream did open
                                    * output stream did open
                                    * output stream has space
                                    * protocol: TLS 1.2
                                    * cipher: ECDHE_RSA_WITH_AES_128_CBC_SHA
                                    * trust result: unspecified
                                    * certificate info:
                                    *  0 rsaEncryption 2048 sha1-with-rsa-signature 's3.amazonaws.com'
                                    …
                                    

                                    Then I wrote a small test app that uses NSURLSession to fetch the root of that site (making sure to disable redirection, as I discussed on 11 Aug).  With the following ATS settings it connected just fine.

                                    <key>NSAppTransportSecurity</key>
                                    <dict>
                                        <key>NSExceptionDomains</key>
                                        <dict>
                                            <key>s3.amazonaws.com</key>
                                            <dict>
                                                <key>NSExceptionAllowsInsecureHTTPLoads</key>
                                                <true/>
                                            </dict>
                                        </dict>
                                    </dict>
                                    

                                    If you have a specific example of it failing, please post it.

                                    Share and Enjoy

                                    Quinn "The Eskimo!"
                                    Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                    let myEmail = "eskimo" + "1" + "@apple.com"

                                      • Re: Why does NSURLSession in iOS9 return -9802?
                                        skunkworks Level 1 Level 1 (0 points)

                                        https://www.dropbox.com/s/2oncfpyg9w2gy2j/TestAppTransportSecurity.zip?dl=0

                                         

                                        Here's a sample project that has NSExceptionAllowsInsecureHTTPLoads.

                                          • Re: Why does NSURLSession in iOS9 return -9802?
                                            eskimo Apple Staff Apple Staff (6,505 points)

                                            Here's a sample project that has NSExceptionAllowsInsecureHTTPLoads.

                                            Thanks.  It’s really helpful to have a simple example to play with.  And, after a bit of spelunking, I have a rough idea what’s going on.

                                            Your test app accesses api.braintreegateway.com, so I compared its TLS setup to dhoerl’s S3 example that I’ve been testing:

                                            $ TLSTool s_client -connect api.braintreegateway.com:443
                                            …
                                            * protocol: TLS 1.2
                                            * cipher: ECDHE_RSA_WITH_AES_128_CBC_SHA256
                                            * trust result: unspecified
                                            * certificate info:
                                            *  0 rsaEncryption 2048 sha1-with-rsa-signature 'api.braintreegateway.com'
                                            …
                                            ^C
                                            $ TLSTool s_client -connect s3.amazonaws.com:443
                                            …
                                            * protocol: TLS 1.2
                                            * cipher: ECDHE_RSA_WITH_AES_128_CBC_SHA
                                            * trust result: unspecified
                                            * certificate info:
                                            *  0 rsaEncryption 2048 sha1-with-rsa-signature 's3.amazonaws.com'
                                            …
                                            ^C
                                            

                                            The results are very similar, with two things to note:

                                            • Your site uses ECDHE_RSA_WITH_AES_128_CBC_SHA256, which is ‘better’ than the ECDHE_RSA_WITH_AES_128_CBC_SHA used by S3.  Both are explicitly allowed by ATS.

                                            • Both sites use sha1-with-rsa-signature, which is the ATS roadblock we’re trying to get around.

                                            I then set up a test project with an ATS dictionary like this:

                                            <key>NSAppTransportSecurity</key>
                                            <dict>
                                                <key>NSExceptionDomains</key>
                                                <dict>
                                                    <key>s3.amazonaws.com</key>
                                                    <dict>
                                                        <key>NSExceptionAllowsInsecureHTTPLoads</key>
                                                        <true/>
                                                    </dict>
                                                    <key>api.braintreegateway.com</key>
                                                    <dict>
                                                        <key>NSExceptionAllowsInsecureHTTPLoads</key>
                                                        <true/>
                                                    </dict>
                                                </dict>
                                            </dict>
                                            

                                            Weirdly, this works for S3 and not for your domain.  Why?

                                            After some digging I worked this out: the OS thinks that your site requires HTTP Strict Transport Security (HSTS).  Now, I don’t know why the OS thinks that, but the fact that it does overrides the ATS exception in the Info.plist.  In short, HSTS trumps ATS.

                                            Alas, I’ve totally run out of time to investigate this in the context of DevForums.  I’d be happy to look into the HSTS side of things in the context of a DTS tech support incident.

                                            Share and Enjoy

                                            Quinn "The Eskimo!"
                                            Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                            let myEmail = "eskimo" + "1" + "@apple.com"

                                          • Re: Why does NSURLSession in iOS9 return -9802?
                                            ifdev Level 1 Level 1 (0 points)

                                            Eskimo, I am seeing the same as Skunkworks. None of the per-domain settings seems to relax the SHA-256 setting except the buggy one.

                                             

                                            Say you have a domain like s3.amazonaws.com, which supports TLSv1.2 and ECDHE, but the RSA 2048-bit cert is signed using sha1WithRSAEncryption. Without ATS exceptions, NSURLConnection fails to connect with kCFStreamErrorDomainSSL -9802 (errSSLFatalAlert).

                                             

                                            But setting NSExceptionAllowsInsecureHTTPLoads=true or NSThirdPartyExceptionAllowsInsecureHTTPLoads=true for that domain doesn't fix it so that  NSURLConnection connects. You still get -9802.

                                             

                                            (THIS PARAGRAPH NOT FACT CHECKED THIS MORNING) Based on some previous experimentation that I haven't retried on the GM, it doesn't fail at the socket layer anymore, and you could implement the various NSURLAuthenticationChallenge related callbacks for NSURLAuthenticationMethodServerTrust to get around the default trust policy that disallows weak certificate fingerprints. But leave me implementing certificate trust myself or figuring out a proper incantation to SecTrustEvaluate. This seems frought with peril. (END OF DISCLAIMED SECTION)


                                            However, if I set NSThirdPartyExceptionRequiresForwardSecrecy=false, suddenly everything just works. In fact, one of my service providers sent an email to their customers on iOS 9 release day telling us they hadn't managed to deploy their sha256 thumbprinted certs and actually suggested including this nonsense exception (nonsense because they do support ECDHE). This was my first inkling that it did not behave as specified in the technote.


                                            Furthermore, it's super scary setting AllowsInsecureHTTPLoads, since the technote says "Use this key to access domains with no certificate, or with an error for a self-signed, expired, or hostname-mismatch certificate." And we do want HTTPS-only. And we don't want to turn on NSAllowsArbitraryLoads because we much prefer a whitelist of known good (or at least necessary) service providers with outdated TLS deployments.

                                              • Re: Why does NSURLSession in iOS9 return -9802?
                                                kharrison Level 1 Level 1 (0 points)

                                                You may want to retest on the public release. I am actually seeing different results between the GM and the final public release which I was not expecting.

                                                 

                                                The Twitter search api at https://api.twitter.com is similar to the Amazon S3 example above. It uses TLS 1.2 and ECDHE_RSA key exchange which provides forward secrecy but has a weakly signed SHA1 signature (see https://www.ssllabs.com/ssltest/analyze.html?d=api.twitter.com&s=199.59.148.87).

                                                 

                                                When testing with the GM I was seeing ATS rejections (error code -9802) which was fixed adding the NSThirdPartyExceptionAllowsInsecureHTTPLoads exception. This would seem to be expected if I have understood the ATS policy correctly.

                                                 

                                                What is odd is that with the public release of iOS 9 the exception is no longer required. The connection succeeds with the default ATS policy. Maybe the requirement for a SHA256 signed signature was relaxed but then at least the ATS Technote should be updated.

                                                 

                                                I have logged radar 22763438 with CFNetwork debug logs for both cases.