NSExceptionRequiresForwardSecrecy and didReceiveAuthenticationChallenge

Hi,



Background: I am working on an application which supports SAML based login. To support SAML based login we open the IDP url in the WKWebView and the IDP take it from there. The IDP based URLs are mostly HTTPS urls and hence they work fine with ATS for our application.



Problem: But unfortunately today we enouctered the problem where the certificate of the IDP is not supported by iOS. The certificate fulfills all requirements except the "Forward Secrecy" requirement. Currently we have "NSAllowsArbitraryLoads" set to NO in our application. Now when we run the application and hit the IDP URL we get the following error in didFailProvisionalNavigation:



didFailProvisionalNavigation::Error Domain=NSURLErrorDomain Code=-1200 "An SSL error has occurred and a secure connection to the server cannot be made." UserInfo={_WKRecoveryAttempterErrorKey=<WKReloadFrameErrorRecoveryAttempter: 0x7f86110935e0>, NSErrorFailingURLStringKey=<URL>, NSErrorFailingURLKey=<URL>, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, NSUnderlyingError=0x7f861109d5d0 {Error Domain=kCFErrorDomainCFNetwork Code=-1200 "An SSL error has occurred and a secure connection to the server cannot be made." UserInfo={_kCFStreamErrorDomainKey=3, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFNetworkCFStreamSSLErrorOriginalValue=-9824, _kCFStreamPropertySSLClientCertificateState=0, NSLocalizedDescription=An SSL error has occurred and a secure connection to the server cannot be made., NSErrorFailingURLKey=<URL>, NSErrorFailingURLStringKey=<URL>, _kCFStreamErrorCodeKey=-9824}}, NSLocalizedDescription=An SSL error has occurred and a secure connection to the server cannot be made.}


I have removed the URL string here.


To fix this I tried implementing the -(void)webView:(WKWebView *)webView didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential *_Nullable))completionHandler method.


Below is the code for the same:


if ([authenticationMethod isEqualToString:NSURLAuthenticationMethodServerTrust]) {

        [ActivityIndicatorHelper hideGlobalHud];

        NSLog(@"protectionSpace::%@", [challenge protectionSpace].description);

        SecTrustRef trust = [[challenge protectionSpace] serverTrust];

        if (trust != NULL) {

            SecTrustResultType secresult = kSecTrustResultInvalid;

            if (SecTrustEvaluate(trust, &secresult) != errSecSuccess) {
                completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
                return;
            }

            NSLog(@"secresult::%u",secresult);
        }

        completionHandler(NSURLSessionAuthChallengeUseCredential, nil);



The above NSLog on line 18 always prints result as "4" which is kSecTrustResultUnspecified which as per definition is obtained only if the certificate is invalid. But in this case it seems that the certificate is "invalid" because it does not fulfill the "Forward Secrecy" requirement. I was banking on the status to be kSecTrustResultRecoverableTrustFailure so that I can use the above method to present the user with an alert with message that warns them that the certificate is not trusted. If they wish to proceed, I will modify the trust setting and let them go ahead. But if I get kSecTrustResultUnspecified which can also be obtained for valid URLs, how can I differentiate between kSecTrustResultRecoverableTrustFailure andkSecTrustResultUnspecified?


To ensure that "forward secrecy" of the certificate is a problem, I modified the ATS setting for my app to be as below and the IDP page started loading properly:


<key>NSAppTransportSecurity</key>
  <dict>
       <key>NSExceptionDomains</key>
       <dict>
            <key>main.app.domain</key>
            <dict>
                 <key>NSIncludesSubdomains</key>
                 <true/>
            </dict>
            <key>idp.vender.domain</key>
            <dict>
                 <key>NSIncludesSubdomains</key>
                 <true/>
                 <key>NSExceptionAllowsInsecureHTTPLoads</key>
                 <false/>  
                 <key>NSExceptionRequiresForwardSecrecy</key>
                 <false/>
            </dict>
       </dict>
</dict>


Is the above behavior of obtaining result as kSecTrustResultUnspecified expected? If yes, how do I differentiate between valid certificate and invalid certificate with "forward secrecy" missing? Any solution is highly appreciated.

Accepted Reply

NSAllowsArbitraryLoadsInWebContent
- This one is available only for iOS 10 and above. And on iOS 10, it doesnt work (this was the first thing I tried). We support iOS 9 and iOS 10.

Yeah, I was worried about that.

NSAllowsArbitraryLoadsInWebContent
does work in most cases but there was a bug that causes the client to not offer non-forward secrecy cypher suites when you use HTTPS (r. 27892687). I believe we fixed that in 10.2. So, things break down as follows:
  • iOS 8 — No ATS to worry about

  • iOS 9 — Rely on

    NSAllowsArbitraryLoads
  • iOS 10.x, x < 2 — I would ignore this case, requiring your users to update to iOS 10.2 or later.

  • iOS 10.2 and later — Rely on

    NSAllowsArbitraryLoadsInWebContent

IMPORTANT The presence of

NSAllowsArbitraryLoadsInWebContent
causes iOS 10 to ignore
NSAllowsArbitraryLoads
. This results in best practice security on iOS 10 while maintaining compatibility with iOS 9.

Why is the connection failing for Forward Secrecy not falling under kSecTrustResultRecoverableTrustFailure … ?

You’re talking about two different subsystems:

  • Cypher suite negotiation is done by the TLS subsystem (CoreTLS > Secure Transport > CFSocketStream (and friends) > NSURLSession)

  • Server trust evaluation is done by way of a trust object (SecTrust)

These are not tightly connected. Specifically, the trust object you get via the

NSURLAuthenticationMethodServerTrust
authentication challenge does not know (or care) what cypher suite was negotiated; it only looks at the certificate chain and the various policies.

If you walk through the process of constructing a trust object (by calling

SecTrustCreateWithCertificates
, feeding it a policy you create using
SecPolicyCreateSSL
), you’ll find that at know point do you tell the trust object what cypher suite was used.

In theory this could happen — we could add a cypher suite policy, or extend the TLS policy to accept the cypher suite as an option — but this is not how things work today.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Replies

The above NSLog on line 18 always prints result as "4" which is

kSecTrustResultUnspecified
which as per definition is obtained only if the certificate is invalid.

You have, alas, got the wrong end of that particular stick.

kSecTrustResultUnspecified
means that the user has not specified any particular preference with regards this certificate. Unless you have specific knowledge that overrides that, you should allow the connection.

To be clear, when looking at

SecTrustResultType
values, you should:
  • Allow the connection if you get

    kSecTrustResultProceed
    or
    kSecTrustResultUnspecified
  • Deny the connection otherwise

Having said that, this is unlikely to be your real problem in this context, because the forward secrecy requirement is enforced by ATS, not by the TLS trust policy (-: If the server has a certificate that’s trusted by the system by default, you can (and should!) completely remove this authentication challenge delegate method.

To ensure that "forward secrecy" of the certificate is a problem, I modified the ATS setting for my app to be as below …

That’s definitely more on track. If you remove

NSExceptionAllowsInsecureHTTPLoads
do things still work? That’s what I’d expect when talking to a server that “fulfills [sic] all requirements except the "Forward Secrecy" requirement”.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi @Quinn,


Thanks for replying. Few things that I should have mentioned earlier are as below:

"Unless you have specific knowledge that overrides that, you should allow the connection.

To be clear, when looking at

SecTrustResultType
values, you should:
  • Allow the connection if you get
    kSecTrustResultProceed
    or
    kSecTrustResultUnspecified
  • Deny the connection otherwise

Having said that, this is unlikely to be your real problem in this context, because the forward secrecy requirement is enforced by ATS, not by the TLS trust policy (-: If the server has a certificate that’s trusted by the system by default, you can (and should!) completely remove this authentication challenge delegate method.


I always want the connection to go through and I dont block it. The reason why I decided to intercept the authentication challenge was to fix the error that I get in didFailProvisionalNavigation. Below is my didReceiveAuthenticationChallenge before I added condition for NSURLAuthenticationMethodServerTrust:


- (void)webView:(WKWebView *)webView didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential *_Nullable))completionHandler {
    NSString *authenticationMethod = [[challenge protectionSpace] authenticationMethod];
    if ([authenticationMethod isEqualToString:NSURLAuthenticationMethodHTTPBasic]) {
        [ActivityIndicatorHelper hideGlobalHud];
        NSString *title = JVLocalizedStringForLabel(@"common.appname", nil);
        NSString *message = JVLocalizedStringForError(@"saml.alert.credentialsMessage", nil);
        UIAlertController *alertController = [UIAlertController alertControllerWithTitle:title message:message preferredStyle:UIAlertControllerStyleAlert];
        [alertController addTextFieldWithConfigurationHandler:^(UITextField *textField) {
          textField.placeholder = JVLocalizedStringForLabel(@"login.placeholder.username", nil);
        }];
        [alertController addTextFieldWithConfigurationHandler:^(UITextField *textField) {
          textField.placeholder = JVLocalizedStringForLabel(@"login.placeholder.password", nil);
          textField.secureTextEntry = YES;
        }];
        UIAlertAction *okAction = [UIAlertAction actionWithTitle:JVLocalizedStringForLabel(@"common.alert.ok", nil)
                                                          style:UIAlertActionStyleDefault
                                                        handler:^(UIAlertAction *_Nonnull action) {
                                                          NSString *userName = ((UITextField *)alertController.textFields[0]).text;
                                                          NSString *password = ((UITextField *)alertController.textFields[1]).text;
                                                          NSURLCredential *credential = [[NSURLCredential alloc] initWithUser:userName password:password persistence:NSURLCredentialPersistenceForSession];
                                                          completionHandler(NSURLSessionAuthChallengeUseCredential, credential);
                                                        }];
        UIAlertAction *cancelAction = [UIAlertAction actionWithTitle:JVLocalizedStringForLabel(@"common.cancel", nil)
                                                              style:UIAlertActionStyleCancel
                                                            handler:^(UIAlertAction *action) {
                                                              completionHandler(NSURLSessionAuthChallengeCancelAuthenticationChallenge, nil);
                                                            }];
        [alertController addAction:okAction];
        [alertController addAction:cancelAction];
        [self presentViewController:alertController animated:YES completion:nil];
    } else {
        completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
    }
}


Unfortunately I still need to implement the delegate method becasue I want to handle NSURLAuthenticationMethodHTTPBasic authentication. But if you see above, for all other authentication types I do completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil); which I guess is default behavior for this method.

To ensure that "forward secrecy" of the certificate is a problem, I modified the ATS setting for my app to be as below …

That’s definitely more on track. If you remove

NSExceptionAllowsInsecureHTTPLoads
do things still work? That’s what I’d expect when talking to a server that “fulfills [sic] all requirements except the "Forward Secrecy" requirement”.


If I remove

NSExceptionAllowsInsecureHTTPLoads
but maintain NSExceptionRequiresForwardSecrecy to FALSE, the URL still loads.


So I think that asking user for confirmation of whether they want to allow the certificate or not (as suggested in the error message in didFailProvisionalNavigation) is not going to work here. The two approaches I see which will work here are as below:


  • Set NSExceptionRequiresForwardSecrecy to FALSE. Can I do this application wide or it has to be done per domain? If its per domain then its going to be very difficult for me to maintain the list as these are 3rd party IDP vendors and we have no control on those.
  • Ask our customer to contact their 3rd party IDP vendor to fix the certificate.


Any suggestions would be greatly appreciated.


Regards,

Bhavik

Ask our customer to contact their 3rd party IDP vendor to fix the certificate.

To be clear, forward secrecy has nothing to do with the server’s certificate. It’s availability depends on the server software. To get forward secrecy the server’s TLS implementation must support one of the ECDHE cypher suites listed in the ATS docs. It’s possible to do this even when the server uses an RSA certificate.

Set NSExceptionRequiresForwardSecrecy to FALSE.

That’s seems like the right approach to me.

Can I do this application wide or it has to be done per domain?

Per domain.

If its per domain then its going to be very difficult for me to maintain the list as these are 3rd party IDP vendors and we have no control on those.

Your only other option is to disable ATS entirely (either for the app as a whole,

NSAllowsArbitraryLoads
, or for the web view,
NSAllowsArbitraryLoadsInWebContent
), which seems like a bad choice. Given that your task is specifically related to security, it might be reasonable to provide forward secrecy exceptions for the common servers and then require additional servers to support forward secrecy properly.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi Quinn,


Thanks for your answer and patience. But I have questions as below:

Your only other option is to disable ATS entirely (either for the app as a whole, NSAllowsArbitraryLoads, or for the web view, NSAllowsArbitraryLoadsInWebContent), which seems like a bad choice. Given that your task is specifically related to security, it might be reasonable to provide forward secrecy exceptions for the common servers and then require additional servers to support forward secrecy properly.


  • NSAllowsArbitraryLoadsInWebContent - This one is available only for iOS 10 and above. And on iOS 10, it doesnt work (this was the first thing I tried). We support iOS 9 and iOS 10.
  • NSAllowsArbitraryLoads - I certainly dont want to do this because this makes our application less secure and moreover, wont the application get rejected when I submit to App Store?
  • Question: Why is the connection failing for Forward Secrecy not falling under kSecTrustResultRecoverableTrustFailure so that I can show an alert to user indicating that the connection is not secure and let them choose. If it falls under kSecTrustResultUnspecified then there is no way for me to distinguish forward secrecy failure and a valid connection. Do you think this should be a recoverable error falling under kSecTrustResultRecoverableTrustFailure or its own type?
  • Is there any other way to recover from this error (since I get this error in didFailProvisionalNavigation) runtime and without the above solutions?


Regards,

Bhavik

NSAllowsArbitraryLoadsInWebContent
- This one is available only for iOS 10 and above. And on iOS 10, it doesnt work (this was the first thing I tried). We support iOS 9 and iOS 10.

Yeah, I was worried about that.

NSAllowsArbitraryLoadsInWebContent
does work in most cases but there was a bug that causes the client to not offer non-forward secrecy cypher suites when you use HTTPS (r. 27892687). I believe we fixed that in 10.2. So, things break down as follows:
  • iOS 8 — No ATS to worry about

  • iOS 9 — Rely on

    NSAllowsArbitraryLoads
  • iOS 10.x, x < 2 — I would ignore this case, requiring your users to update to iOS 10.2 or later.

  • iOS 10.2 and later — Rely on

    NSAllowsArbitraryLoadsInWebContent

IMPORTANT The presence of

NSAllowsArbitraryLoadsInWebContent
causes iOS 10 to ignore
NSAllowsArbitraryLoads
. This results in best practice security on iOS 10 while maintaining compatibility with iOS 9.

Why is the connection failing for Forward Secrecy not falling under kSecTrustResultRecoverableTrustFailure … ?

You’re talking about two different subsystems:

  • Cypher suite negotiation is done by the TLS subsystem (CoreTLS > Secure Transport > CFSocketStream (and friends) > NSURLSession)

  • Server trust evaluation is done by way of a trust object (SecTrust)

These are not tightly connected. Specifically, the trust object you get via the

NSURLAuthenticationMethodServerTrust
authentication challenge does not know (or care) what cypher suite was negotiated; it only looks at the certificate chain and the various policies.

If you walk through the process of constructing a trust object (by calling

SecTrustCreateWithCertificates
, feeding it a policy you create using
SecPolicyCreateSSL
), you’ll find that at know point do you tell the trust object what cypher suite was used.

In theory this could happen — we could add a cypher suite policy, or extend the TLS policy to accept the cypher suite as an option — but this is not how things work today.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi Quinn,


Thanks for your patience and explanatory replies.

In theory this could happen — we could add a cypher suite policy, or extend the TLS policy to accept the cypher suite as an option — but this is not how things work today.


Do you think this can be an enhancement that can be considered? A radar would help?


Regards,

Bhavik

Do you think this can be an enhancement that can be considered?

Your question caused me to sit down and think about this some more, and on reflection I don’t think this will work. Or, more specifically, some parts of ATS’s enhanced security checks could move into a trust policy object, but not all of it, and certainly not the part that you care about (forward secrecy).

ATS’s enhanced security includes four additional checks above and beyond those done by normal RFC 2818 server trust evaluation:

  • The server must accept a TLS 1.2 connection (A)

  • The TLS cypher suite must be in a specific list (B)

  • The key in the server’s certificate must meet specific criteria (C)

  • The server’s certificate must be secured by a modern hash (D)

Of these, C and D are done after receiving the TLS Certificate message from the server, and thus would fit reasonably into the trust evaluation mechanism. However, A and B are different:

  • For A, the client should drop the connection as soon as it receives the TLS Server Hello message (which is where the client learns about the server’s TLS version choice), not on the TLS Certificate message, which comes next. This probably isn’t a big deal, but it’s definitely a change of behaviour.

  • For B things are trickier. The client must know in advance whether it’s going to require forward secrecy because, if it is, it should only put forward secrecy capable cypher suites in the TLS Client Hello message. The alternative would be for the client to include both types of cypher suites in the TLS Client Hello message, and then drop the connection after getting the TLS Certificate message if the server has chosen the ‘wrong’ one. That seems like very poor form in general, and it could cause connections to fail.

    Imagine a server that supports some cypher suites F and NF, where F is a cypher suite that supports forward secrecy and NF is one that doesn’t. However, for whatever reason, the server prefers NF over F. In the current setup the client will only offer F, which the server allows, and all is good. If we change things so the client offers both F and NF and then fails the connection if the server chooses NF, the connection won’t work.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi Quinn,


Thanks for your detailed and inshightful reply as always. To close the thread, below is the solution I went ahead with:


- Implement domain based ATS.

- Enable ATS and Forward Secrecy for the domain (in our control) to which our application connects for all API calls.

- Enable ATS for the specific third party IDP domain (not in our control).

- Disable Forward Secrecy for the specific third party IDP domain (not in our control).

- We gave a timeline to our customer to update their IDP server to support forward secrecy.


The problem with above soluation is that we will have to "whitelist" each and every domain that our customers would report not working because of forward secrecy. But so far we have heard only from 1 customer and we gave them a time frame to add support for forward secrecy. We might change this approach to what you suggested above if more and more customers report similar problem.


Regards,

Bhavik

Getting NSURLError (TIC Read Status [12:0*0]: 1:57) in XCode for the iOS App running on windows server without TLS 1.2 & Forward Secrecy setup. After going through the iOS security guide, we enabled TLS 1.2. After that also we are getting the same error message. Is Forward Secrecy mandatory? Is Forward Secrecy the reason for this issue? Please help.