iOS 11.3: WKWebView cookie synchronizing stopped working

Hi,


Anyone else having issues with WKWebView cookies after iOS 11.3? (Or any recomondations of what we might be doing wrong?)


Working on an app with web content, we switched fro UIWebView to WKWebView after iOS 11.

We use URLSession for authentication, then webviews to show content.


In order to syncrhonzie cookies to WKWebViews, we have used the following procedure:


1. Log in with URLSession

2. Create a WKWebView with the cookies from the URLSession:

let config = WKWebViewConfiguration()
config.processPool = WKProcessPool()
let cookies = HTTPCookieStorage.shared.cookies ?? [HTTPCookie]()
cookies.forEach({ config.websiteDataStore.httpCookieStore.setCookie($0, completionHandler: nil) })
          
let wkWebView = WKWebView(frame: bounds, configuration: config)
...


3. Load the URL.


Worked like a charm, until iOS 11.3.


Something fishy is happening, I have checked that the HTTPCookieStorage contains the correct cookies, and verified that the completionHandler of "setCookie" gets called before the url loads. However, the cookies are not being set.


But. If we wait with creating the WKWebView/transfering cookies by about ~3 seconds after loggign in, it seems to work.


This might seem like a bug introduced in iOS 11.3 (?), or any other suggestions?

Thanks for your input!


We also solved our problem by calling the websiteDataStore before creating the webview, seems like the commit you are talking about might be the problem.


We have kind of the same flow, as mentioned in the main post:


1. (native) login with URLSession

1,5: "Hack": call websiteDataStore

2. transfer cookies from HTTPStorage to WKWebView

3. navigate to URL in WKWebview that requires a cookie set


If we don't do step 1,5, the cookies for the WKWebView in the "decidePolicyFor" delegation method is wrong


if #available(iOS 11.3, *) {
    let _ = WKWebViewConfiguration().websiteDataStore
}


Saw the ticked you made @hshamanskygot an update a few days ago. We have ***** with the latest version of iOS 12 (beta 4) - seems like the issue is still there.


Any updates on your finds @eskimo?


The sad part for us, is that it seems like this will fix the issue 99% of the times. It's hard to tell when it fails, and we can't reproduce it, it just fails now and then.

Any updates on your finds @eskimo?

I had a look at the various bugs clustered around this issue and none of them indicate a clear fix. If you have a workaround that works for you, that’s cool. In my testing I was able to find various workarounds that work in various circumstances, but nothing universal )-:

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Okey – I guess we'll live with the workaround for now, and hope for a fix in the future. Thanks for your update 🙂

I can confirm this problem still exists in the latest iOS12 Release version.


Our worklfow is quite simple:

We basically have a viewcontroller with a webview.

From a second viewcontroller we navigate to the webview-viewcontroller.

There we call "webview.configuration.websiteDataStore.httpCookieStore.setCookie(***)"

After its completion, we load the webview.



So I went trough a lot of debugging and setting breakpoints and what i found out: on the first appstart (quit app in backround and make a fresh start), when the viewcontroller with the webview is loaded the first time, the following happens:


- webview.configuration.websiteDataStore.httpCookieStore is empty, which is correct

- we call webview.configuration.websiteDataStore.httpCookieStore.setCookie(***) to set our cookie

- the completion handler gets called.

- after short delay the "func cookiesDidChange(in cookieStore: WKHTTPCookieStore)" gets called, which is also expected


- so with the completionhandler of "setCookie" complete and even the observer "cookiesDidChange" triggering, we can expect that the cookie was indeed set. So now we check the content of the "webview.configuration.websiteDataStore.httpCookieStore" and what does it show? -> It's empty. No cookies where set even tough the completion handler was called and "cookiesDidChange" reported a change.


This only happens on fresh app starts and the first time the above process is triggered. Even when the webview and its viewcontrollers get dismissed later and reinstanciated with the same process, setting the cookies will work as expected. So i guess there is indeed some problem with cookie storage not beeing initiated correctly from the system on a fresh app start until the first time its actually used in some way. And while the "setCookie" completionhandler fires and the "cookiesDidChange" gets triggered, the cookies is never really set.


I can also confirm that the mentioned workaround seems to help. So what we do now is: When the viewcontroller with the webview gets initiated, in viewWillAppear we call "webview.configuration.websiteDataStore.httpCookieStore.getAllCookies()". We dont store the returned values we just call the method. Just making this call seems to prevent the above problem to appear and the cookies will get set as expected.

I can confirm this problem still exists in the latest iOS12 Release version.

It sounds like you’ve done a really good job of isolating this problem. If you can boil that down into a small test project and attach that to a bug report, I’d be most grateful.

Please post your bug number, just for the record.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi, i will do so as soon as possible.


But I can give an update to my workflow:

It seems like my solution by calling "webview.configuration.websiteDataStore.httpCookieStore.getAllCookies()" did not work as reliable as it should have. So I overhauled my code and at least fo now, it seems like i finally found a solution for my situation:


It seems like it makes a difference if you try to add cookies to an existing WKWebview or if you create one after setting the cookies. Somewhere on stackoverflow people reported that an already existing WKWebview sometimes ignores set cookies (at least on a fresh appstart)


So we are now using the following workflow (code simplified):

//Instead of adding the cookies to an already existing instance of a WKWebview
//We create the webview only !after! setting all cookies:

//Create a new WKWebViewConfiguration and a new WKWebsiteDataStore
let config = WKWebViewConfiguration()
let wkDataStore = WKWebsiteDataStore.nonPersistent()

//Get sharedCookies from HTTPCookieStorage
let sharedCookies: Array<HTTPCookie> = HTTPCookieStorage.shared.cookies(for: webviewURL) ?? []
let dispatchGroup = DispatchGroup()

//Add each cookie to the created WKWebsiteDataStore
//Wait until all setCookie completion handlers are fired and proceed
if sharedCookies.count > 0 {
    //Iterate over cookies in sharedCookies and add them to webviews httpCookieStore
    for cookie in sharedCookies{
        dispatchGroup.enter()
        wkDataStore.httpCookieStore.setCookie(cookie){
            dispatchGroup.leave()
        }
    }
    
    //Wait for dispatchGroup to resolve to make shure that all cookies are set
    dispatchGroup.notify(queue: DispatchQueue.main) {
      //Add datastorage to configuration
      config.websiteDataStore = wkDataStore
      //Now crate a new instance of the wkwebview and add our configuration with the cookies to it
      self.webview = WKWebView(frame: self.view.bounds, configuration: config)
      self.webview!.navigationDelegate = self
            
      self.view.addSubview(self.webview!)
      self.webview!.load(request)
    }
}
else{
     //do something else
}


For now it seems like the above workflow finally brought the solution.

This is a bit of a retrospective, but we have just been bitten terribly by the bug/change. We read a million articles / this forum / stackoverflow questions that all seemed to stem from cookie behavior changing in 11.3/11.4/12+, I've read this forum dozens of times. Our app got hammered by negative reviews, as users continuously kept getting logged out as the cookies stopped working consistently, and users were tweeting and calling our support center. Our QA process could not reproduce what was going on, we haven't pushed an App Store update to our app in months. Sometimes it worked, sometimes force closing worked, sometimes when you logout/login-as-another-user, then force close, and re-enter the app, you were logged in as the previous user, chaos...


Tangent: I'd like to complain that in iOS that there isn't just one best "WebView" class. It got splintered into UIWebView, and WkWebView, and app developers had to deal with intricacies related to managing networking, and cookies with each implementation. (Android did have a parallel chrome webview, but it got merged into just WebView, and networking and cookies are all managed by the OS, nobody had to carry an albatross of code managing which-webview-on-which-version-and-how-to-migrate). Since we want to support users with as many versions of iOS, we chose to build a WebViewAbstraction that allowed us to use the best WebView implementation per OS version. We then also synced cookies back to UserDefaults so that in case the user upgrades iOS versions, we would have a way of reading and migrating cookies. We then had to overload cookiesDidChange, we had to manage our cookie stores. And this rube-goldberg machine worked until iOS 11.3/11.4/12.0. Without any big announcement, that I'm aware of, very subtley, users who upgrade iOS, get random logout/cookie issues. Impossible to consistently reproduce.


Here's some psuedo-code of what our app used to be:

#onStart
if(iOS >= 11) {
  webviewWrapper = WKWebView()
  readCookiesFromUserDefaultsIntoWKWebView()
} else {
  webviewWrapper = UIWebView()
  readCookiesfromUserDefaultsIntoUIWebView()
}

// Our cookies expire after an hour, and the app-server sends updated cookies, by consuming a token
cookiesDidChange() {
  syncCookiesFromWebViewToUserDefaults()
}

#onStop
saveCookiesFromWebviewToUserDefaults()


We also have HTTPS requests that our app has to make, and we have to wire it up with the correct session/cookieStore for which webview we're using. And when those HTTP requests complete, they then have to write new cookies into the cookieStore.


Our Solution:

- Drop support for iOS < 11. The WkWebView has presentation bugs in iOS < 11. The UIWebView has presentation bugs galore, and doesn't work well with our HTML app. And this whole debacle made clear that managing our own cookies is frought with adversity. We then removed the syncing to/fro UserDefaults. We removed UIWebView, we removed the WebViewWrapper, we removed our CookieStoreAbstractor. Very vanilla WkWebView inside our ViewController. We don't loadCookies, we don't setCookies, we don't cookiesDidChange, none of it.


When our HTTP requests have to read current cookies I manually construct a http header "cookieOne=1; cookieTwo=2". When the HTTP request responses have something to write to cookies, thats the only piece of magic we have:


let allHeaderFields = httpResponse.allHeaderFields as! [String : String]
let cookies = HTTPCookie.cookies(withResponseHeaderFields: allHeaderFields, for: URL(string: self.configuration.environment.cookieDomain)!)

// Need to run writing cookies on the "main thread". Load homepage when all writes are done (enter/leave/notify).
DispatchQueue.main.async {
  let dispatchGroup = DispatchGroup()

  for cookie in cookies {
    dispatchGroup.enter()
    webView.wkWebView!.configuration.websiteDataStore.httpCookieStore.setCookie(cookie, completionHandler: {
      dispatchGroup.leave()
    })
  }

  dispatchGroup.notify(queue: .main) {
    // load the home page into the web view.
    webView.loadURL(urlRequest: URLRequest(url: URL(string: self.configuration.environment.mainURL)!))
  }
}


TL;DR; Custom cobbling a cookie solution is broken on iOS 11.3/11.4+, so we fixed it by removing everything, and just using WkWebView default, and our cookies aren't broken any more...


Its been a terrible couple of weeks of working all day and all night trying to find out what went wrong, where are the race conditions, and exploring every possible solution, until we just looked at what would be the simplest solution, and it involved nuking everything. I hope this post helps the next person.

The problem with syncing cookies to WKWebView lies in WKProcessPool. To properly sync cookie, you have to create an instance of WKProcessPool and set it to the WKWebViewConfiguration that is to be used to initialize the WkWebview itself:


    private lazy var mainWebView: WKWebView = {
        let webConfiguration = WKWebViewConfiguration()
        if Enviroment.shared.processPool == nil {
            Enviroment.shared.processPool = WKProcessPool()
        }
        webConfiguration.processPool = Enviroment.shared.processPool!
        webConfiguration.processPool = WKProcessPool()
        let webView = WKWebView(frame: .zero, configuration: webConfiguration)
        webView.navigationDelegate = self
        return webView
    }()


Setting WKProcessPool is the most important step here. WKWebview makes use of process isolation - which means it runs on a different process than the process of your app. This can sometimes cause conflict and prevent your cookie from being synced properly with the WKWebview. If you don't use the same instance of WKProcessPool each time you configure a WKWebView for the same domain (maybe you have a VC A that contains a WKWebView and you want to create different instances of VC A in different places), there can be conflict setting cookies. To solve the problem, after the first creation of the WKProcessPool for a WKWebView that loads domain B, I save it in a singleton and use that same WKProcessPool every time I have to create a WKWebView that loads the same domain B


After the initialization process, you can load an URLRequest inside the completion block of httpCookieStore.setCookie. Here, you have to attach the cookie to the request header otherwise it won't work.


    mainWebView.configuration.websiteDataStore.httpCookieStore.setCookie(your_cookie) {
            self.mainWebView.load(your_request, with: [your_cookie])
    }


    extension WKWebView {
       func load(_ request: URLRequest, with cookies: [HTTPCookie]) {
          var request = request
          let headers = HTTPCookie.requestHeaderFields(with: cookies)
          for (name, value) in headers {
             request.addValue(value, forHTTPHeaderField: name)
          }      
          load(request)
       }
    }

eskimo,


If you have a working sample, can you please share your code sample? Thanks!

Hello!


I finally solve this problem successfully,



I used the webView navigationResponse: WKNavigationResponse to get cookies and save them to local storage, and every time the application started, I consulted the localstorage to get saved cookies and inject them into httpCookieStore before the first load of the webView



I hope this can help sameone.



Att,

Khalid Ait Bellahs

https://www.tialtonivel.com.br

Thank you!


I've been struggling with an intermittent dropping of session cookies for ages, and using a single WkProcessPool as per your first piece of code appears to have solved it (I haven't seen a drop in 2 weeks now)

Hi guys, I am seeing something similar in an App that I am working on where there seems to be an intermittent problem with the expected Cookies being available in the WKWebView's cookie store. Is there a consensus that forcing the default WKWebsiteDataStore class into action by calling a method on it might avoid problems caused by lazy initialisation ?
Hi Quinn,

I’ve been tracing the cookie issues for over a month now and I am
able to reproduce and remedy all test cases 100%.

I do have a dedicated test environment with utility buttons in the UI
that let me e.g. log the cookieStore, empty the cookieStore etc.

All WKWebview delegate methods and cookieStore observers are
in place and log to the console when they fire.


So the 2 take away test scenarios for you are these:


———————————————————
SCENARIO 1:
———————————————————


• starting the test environment
• totally resetting the cookieStore by deleting all cookies
• logging the cookieStore shows that it is empty
• quitting the test environment


• re-loading the test environment
• environment is loading a test webpage from the internet with:
myWebview.(URLRequest)
• the page loads correctly
• the response-header’s ‘Set-Cookie’ field requests 3 cookies to be set
• the WKWebview sets 3 cookies (as requested in the response-header’s ‘Set-Cookie’ field)
• the didChangeCookieStore callback fires 3 times
• logging the cookieStore shows that there are 3 cookies
• the 3 cookies are:
— 1 permanent cookie (with expiration date set) and
— 2 temporary session cookies (without expiration date set) -> [csrfToken, CAKEPHP] tokens


———————————————————
Problems:
———————————————————


NONE !!!


———————————————————
Observations:
———————————————————


Unfortunately the logged response header DOESN’T display the ‘Set-Cookie’ field !!!


=====> WK-WEB-VIEW: decidePolicyFor navigation RESPONSE

<WKNavigationResponse: 0x160a13df0; response = <NSHTTPURLResponse: 0x160a1a990>
{ URL:https://testURL }
{ Status Code: 200, Headers {...}}


———————————————————
SCENARIO 2:
———————————————————


• totally resetting the cookieStore by deleting all cookies
• logging the cookieStore shows that it is empty
• quitting the test environment


• re-loading the test environment
• environment is loading a test webpage from the internet with
URLSession.shared.dataTask(with: URLRequest) { data, response, error in ... }
• the response-header’s ‘Set-Cookie’ field requests 3 cookies to be set
• the didChangeCookieStore callback fires 3 times
• logging the cookieStore shows that there is only 1 cookie
• the 1 cookie is:
— 1 permanent cookie (with expiration date set)
— NONE of the 2 expected temporary session cookies (without expiration date set) have been set -> [csrfToken, CAKEPHP] tokens


———————————————————
Problems:
———————————————————


Of course, if you now load the mime ‘text/HTML’ payload
(by casting the response’s data object into a string) with

• myWebview.loadHTMLString()

all user actions on that HTML page like e.g. triggering hidden requests
in POST forms that rely on the [csrfToken, CAKEPHP] tokens etc., are
denied by the server because of the missing cookies.

Equally, if there is e.g. an additional ajax HTTP request in this HTML’s
onLoad() function, I can see in the server logs, that this additional request
comes without the 2 [csrfToken, CAKEPHP] session cookies attached
(only the single permanent cookie is present) – and so the server re-sends
a new pair of [csrfToken, CAKEPHP] cookies with the new response.

As this ajax request/response only updates some minor data in the UI, but
the DOM is still the the same from the initial page request, you can imagine
that the new [csrfToken, CAKEPHP] cookies do not match up with all the
references in the HTML either, because of ‘token mismatches’.

The page is still broken!


———————————————————
Observations:
———————————————————


As the problems only seem to affect temporary session cookies maybe the bug
trace should focus on that area. Remember - every scenario was started with a
completely wiped out empty cookie store!


———————————————————


Fortunately the logged response header DOES display the ‘Set-Cookie’ field !!!


=====> WK-WEB-VIEW: decidePolicyFor navigation RESPONSE

RESPONSE: <NSHTTPURLResponse: 0x110172520> { URL: https://testURL } { Status Code: 200, Headers {

    "Set-Cookie" =     (

        "CAKEPHP=bjhbajljh9f78vm9mko836kk2h; path=/; secure; HttpOnly",

        "interfaceLanguage=deu; expires=Sat, 15-May-2021 16:21:10 GMT; Max-Age=2591999; path=/",

     "csrfToken=591d618d10632e05f1aa79e003e82a6c5da73781f55466782885...; path=/"

    );
} }


———————————————————
Miscellaneous:
———————————————————


When the navigationAction.didFinish delegate fires, thecookieStore isn’t
properly set yet, despite the didChangeCookieStore observerver having
fired long before.

You have to wait about 1.75 seconds in the best case scenario, but often
up to 2.5 seconds. So right now 3 seconds seems to be a safe setting.

I would rather expect the didFinish delegate to fire when even the
asynchronous cookieStore is in a valid state, or at least have an additional
delegate method for when really everything is completed...


Needless to say, that I did all cookieStore checks manually with a utility
button in my test environment UI, waiting at least 5 seconds to be sure
that all asynchronous tasks have finished.


———————————————————
REMEDIES:
———————————————————


One remedy for the missing cookie issue is to stash away the ‘Set-Cookie’
header from the initial request, extract the tokens from the signature,
instantiate some custom cookies with the respective values and store
them into the cookieStore.

But that is a little bit of a management nightmare!


———————————————————
GENERAL OBSERVATION AND
ENHANCEMENT REQUEST

———————————————————

Unfortunately the system doesn’t inform you when the server responds with a
REDIRECT 302 status. Neither with a <decidePolicyFor navigation RESPONSE>
nor with a dedicated callback. The system rather decides to issue a new request
automatically and you only find yourself with an additional invocation of
<decidePolicyFor navigation ACTION> wondering where that came from,
because there is no mention of any status 302 in the whole processing and
logging chain.

If I hadn’t had access to the server logs I would have never known about the
302, and kept on wondering, what in my code triggers this additional request.

However, I have seen in the WKNavigationAction.h file that there is a property
called ’isRedirect’ just along with ‘navigationType’ etc. that one can access via

• navigationAction.value(forKeyPath: "isRedirect")

In the case of the automatic REDIRECT that the system performs, this value is
set to 1 for the following <decidePolicyFor navigation ACTION>.

It would be really helpful if the navigationAction.description would make this
information available along with

• <WKNavigationAction: 0x112076910;
— navigationType = -1;
— syntheticClickType = 0;
— position x = 0.00 y = 0.00

— isRedirect = 1

— request = ...


without having to explicitly poll for it.


———————————————————


ADDENDUM:

One thing that would clarify this problem, however not solve it, is the fact that a plain

myWebview.load(URLRequest)

is using Mozilla/5.0 as the agent, which is confirmed by the server log.

However

URLSession.shared.dataTask(with: URLRequest) { data, response, error in ... }

uses a CFNetwork executionExtension as the agent, also confirmed by the server log.


So if the executionExtension spins up its own URLSession that writes the cookies to
the cookieStore, it could be that after the dataTask’s completion handler the session
gets deallocated and hence the temporary session cookies get removed from the
cookieStore right away.

The question then is, how do you get a copy of those short-lived cookies before the
extension closes, because even

myWebview.configuration.websiteDataStore.httpCookieStore.getAllCookies { cookies in ...}

is asynchronous and too slow to grab hold of them. And even when
loading the response’s mime ‘text/HTML’ payload with

• myWebview.loadHTMLString()

right in the completion handler, it is too slow to find the cookies it would
need to e.g. attach to an ajax request in the onLoad() function of the
loaded HTML page.

Not to mention the general out-of-sync situation of all the forms in the
HTML that rely on corresponding tokens stored in cookies in order to
make POST requests etc.



———————————————————


WKWebView is a asynchronous component, UIWebView is a synchronous component....is there any option to force synchronous ?

iOS 11.3: WKWebView cookie synchronizing stopped working
 
 
Q