WatchOS: background refresh file is downloaded but didFinishDownloadingTo: never called...

Greetings,
I've implemented background refresh in my WatchOS app, its purpose is to get some data (always less than 400 bytes) from the server at a minimum of every 15 minutes and update the complication.

Sounds pretty straight forward and it's like clockwork - works really well when I'm debugging the WatchOS app in Xcode or started by launching it manually.

The problem begins if I do something to force the system to kill the app by simply swiping through all the watch faces, or loading other apps. This is a highly realistic scenario.

At this point I switch to observing what is happening on the watch by viewing the console output:
  1. WatchOS launches my app in the background and my extension's handle() method is called

  2. This in turn invokes my code which creates a properly crafted URLRequest which is then passed to a background URLSession's download task.

  3. The very small (<400 bytes) file is downloaded. This works every time, regardless of how the app is launched.

  4. My URLSessionDownloadDelegate's urlSession(_ downloadTask:didFinishDownloadingTo:) method is NOT called, even though the file has been downloaded successfully!

  5. Even stranger, the delegate method IS then called only if I then tap on my complication which launches the app.


Please tell me that this is not how the intended functionality is supposed to work. How am I supposed to update the complication in the background if the only time I can process the downloaded data is if the user starts the app?

I was under the (hopefully not mistaken) impression that if I have my app's complication on the Watch face I would be able to process the result and update the complication data in the background. Is that a wrong impression?

If I am mistaken, then the complication cannot be updated 4 times per hour, and what is the point then of downloading the data in the background if the user then has to tap on the stale complication?

Thanks in advance,
Sal


Answered by sal_from_new_york in 636388022
After much testing on both WatchOS 6 & 7 here's what I've determined:
  1. WatchOS 7 appears to have a better scheduling algorithm than WatchOS 6. Can't prove it though.

  2. If my app has died and WatchOS starts it, although the scheduling of the background refresh still happens, the URLSessionDownloadDelegate's urlSession(_ downloadTask:didFinishDownloadingTo:) method is NOT called until the user either taps on one of my complications or starts the app from the home screen.

  3. The times which I've observed  skips in refresh, and my complication now says it's been over 15 minutes long is because a network error such as a timeout has occurred.

  4. It appears that keeping a reference to the latest URLSession is worth doing, just ensure that you remove the reference when it has completed successfully or otherwise.

  5. Using earliestBeginDate had a detrimental effect on my app. It would keep getting terminated by the system as soon as I resumed the first URLSession.

Although it is stated in the documentation that the OS will retry the request if a network error has occurred, I can't determine if that's true since I don't know how many times it actually retried.

Same issue with my app.

I can see from the logs that just before the error, system is looking at the Info.plist file of the app and then notifies about the "not support BAR" -- should I have something in the Info.plist to tell the system the app supports launching from the background?

Code Block
unixio debug 20:21:49.342657+0300 com.apple.securityd Carousel open(/private/var/containers/Bundle/Application/C8EF6D34-73C3-45C7-A669-0E501F622A44/Xxxx WatchKit App.app/Info.plist,0x0,0x1b6) = 37
workspace error 20:21:49.356798+0300 com.apple.Carousel Carousel com.myorg.MyApp.watchkitapp: app does not support BAR, not creating action for reason CSLSDuetActivationReasonComplication -- openApplicationOptions: {
CSLSDuetActivationServiceReasons = (
CSLSDuetActivationReasonComplication
);
"ActivateSuspended" = 1;
"PayloadOptions" = {
budgetdictionary = "/watch/budget/launches";
kCSLBackgroundActivityUUID = "089EE1D6-7B07-46AE-9889-90847D350D99";
kCSLBackgroundActivityUserString = "E4D6101A-7ACD-495F-AB7C-6571A042DA48";
};
}


Hi Andy,
In your WatchOS app extension's Signing and Capabilities section do you have a "Background Modes" section? I do. Additionally I have session type set to "None" and for Modes I have "Remote Notification" checked. I believe that is all you need.

What on earth is "BAR"?

The issue is that for me, background processing occurs only if the user has started my app and it now is in the application dock. If the app is dead, although it gets started by WatchOS at the desired time, it cannot process the data file until the user brings it to the foreground. I just want to know if that is the intended functionality or if I'm missing something.
I do not have the background modes on, since my app does not fulfill those use cases listed there. My app just needs to download a small JSON every 30-60 minutes and process it (environmental data). Reading the docs and watching the videos on the topic do not explicitly mention anywhere that background processing should be set nor required. Unless your app runs "constantly".

Anyways, after extension is woken up to handle the background processing, the URLSession task is handled in the background in nsulrsessiond daemon when requested (after say 30 mins) and when the download task completes, system should wake up the app to handle the downloaded content.

Though I am starting to suspect that this is not the case because of that log error I posted. Alternatively (and hopefully), the system yet does not yet support this but will in the final watchOS 7.

My interpretation is that background modes are not needed, but the watch face must have a complication from your app and/or it should be in the doc (recently used) to background URLSession downloads to happen as requested. And even then, system may decide that app requests are not processed according to the schedule (battery low, bad network, app has made too many requests or using too much CPU, ....

I do not know where "BAR" comes from, but it is used in one of the WWDC2020 videos in the sample code. And since it is visible in the logs, it must be something the system "knows". Haven't found any explanation of what it is about. Sample code used the term BAR too, e.g. in my code where the function name is from the sample code:

Code Block
   private func scheduleBAR(_ first: Bool) {
      logger.debug("ExtensionDelegate: scheduling background data refresh.")
      let now = Date()
      let timeInterval = first ? 5.0 : 15.0 * 60.0
      let scheduledDate = now.addingTimeInterval(timeInterval)
      let info: NSDictionary = ["submissionDate": now]
      let wkExt = WKExtension.shared()
      wkExt.scheduleBackgroundRefresh(withPreferredDate: scheduledDate, userInfo: info) { (error: Error?) in
         if error != nil {
            self.logger.error("background refresh could not be scheduled: \(error.debugDescription)")
         } else {
            self.logger.info("background refresh scheduled successfully after \(timeInterval) secs")
         }
      }
   }

I believe that BAR stands for Background Application Refresh. Therefore the snippet you posted from WWDC is for scheduling one. I'm also pretty certain that you need to at least add background session type set to "None" to your app's signing and capabilities and that is what the error in the console is complaining about.

I have mine set to process remote notifications but it doesn't have to be set to anything, especially if you don't need to use an extended runtime session. Also the session type has a choice for "None" which represents a generic background refresh session that you want to create. Your app doesn't have to have an extended background runtime session, it just needs to declare that it uses a background refresh of a generic type ("None")

Why not give it a try? The worst that will happen is nothing
Thanks Sal, I'll give it a try and report here what happens.
No luck. Still refusing to launch my extension/bg functionality (see log below at line 12). I also noticed this error in the logs:

Code Block
+[WKExtension sharedExtension]:38: Extensionless WatchKit apps should use WKApplication


Which is also discussed in another thread. Maybe this is related, I do not know.

But your problem is different, you succeed in launching the background request but then the URLSessionDownloadDelegate's urlSession is not called. So my comments do not help solving your issue. I'll try out something else. Thanks for your suggestions though!

Code Block
Common default 13:22:43.829113+0300 com.apple.FrontBoard dasd [FBSSystemService][0x8a65] Sending request to open "com.mydomain.myapp.watchkitapp"
Common default 13:22:43.830375+0300 com.apple.FrontBoard Carousel [FBSystemService][0x8a65] Received request to open "com.mydomain.myapp.watchkitapp" from dasd:91.
workspace tiedot 13:22:43.830965+0300 com.apple.Carousel Carousel handleOpenApplicationRequest for com.mydomain.myapp.watchkitapp from dasd
workspace default 13:22:43.831234+0300 com.apple.Carousel Carousel com.mydomain.myapp.watchkitapp: Duet activation requested by <FBProcess: 0x1617b3f0; daemon<com.apple.dasd>:91(v17E)> for reasons: (
CSLSDuetActivationReasonComplication
)
workspace default 13:22:43.831532+0300 com.apple.Carousel Carousel com.mydomain.myapp.watchkitapp: Duet initiated launch allowed to proceed
unixio debug 13:22:43.835551+0300 com.apple.securityd Carousel open(/private/var/containers/Bundle/Application/C84E1395-6686-46DA-AD72-10C8FDEF7AAE/My WatchKit App.app/My WatchKit App,0x0,0x1b6) = 36
unixio debug 13:22:43.835742+0300 com.apple.securityd Carousel open(/private/var/containers/Bundle/Application/C84E1395-6686-46DA-AD72-10C8FDEF7AAE/My WatchKit App.app/My WatchKit App,0x0,0x1b6) = 37
machorep debug 13:22:43.836451+0300 com.apple.securityd Carousel 6676 signing bytes in 5 blob(s) from /private/var/containers/Bundle/Application/C84E1395-6686-46DA-AD72-10C8FDEF7AAE/My WatchKit App.app/My WatchKit App(arm64_32)
unixio debug 13:22:43.841957+0300 com.apple.securityd Carousel open(/private/var/containers/Bundle/Application/C84E1395-6686-46DA-AD72-10C8FDEF7AAE/My WatchKit App.app/Info.plist,0x0,0x1b6) = 36
workspace error 13:22:43.844990+0300 com.apple.Carousel Carousel com.mydomain.myapp.watchkitapp: app does not support BAR, not creating action for reason CSLSDuetActivationReasonComplication -- openApplicationOptions: {
CSLSDuetActivationServiceReasons = (
CSLSDuetActivationReasonComplication
);
"ActivateSuspended" = 1;
"PayloadOptions" = {
budgetdictionary = "/watch/budget/launches";
kCSLBackgroundActivityUUID = "A3368012-79B9-4170-9008-EA7D0A23651F";
kCSLBackgroundActivityUserString = "F4FE89A5-28A9-4F2D-A78E-B8F8B0263D1F";
};
}


Finally got this working. What I needed to do:
  1. Not use the new SwiftUI app architecture, but implement a class HostingController: WKHostingController<AppMainView>

  2. Include a Interface.storyboard in your watch app (not extension) which refers to the HostingController as the main scene.

Code Block <scenes>
        <!--Hosting Controller-->
        <scene sceneID="Kua-8L-UD8">
            <objects>
                <hostingController id="E91-za-DJq" customClass="HostingController" customModule="YourApp_WatchKit_Extension"/>
            </objects>
            <point key="canvasLocation" x="-35" y="125"/>
        </scene>



Then, in your Extension, initiate the background request e.g. when the app is put to background, using WKExtension.scheduleBackgroundRefresh().

When handling the background task request in Extension, first repeat the request (if you need periodical updates), and initiate your URLSession background request as usual:

Code Block
         case let backgroundTask as WKApplicationRefreshBackgroundTask:
            scheduleBAR(false)
            YourAppDataProvider.shared.fetch(false) // initiates the URLSession background request here


The YourAppDataProvider is a URLSessionDownloadDelegatehandling the downloading. Also make sure Extension gives the urlsession task to the provider to handle when it comes:

Code Block
         case let urlSessionTask as WKURLSessionRefreshBackgroundTask:
            YourAppDataProvider.shared.handleDownload(urlSessionTask)


And there, refresh the session in case the app/extension was closed while in background:

Code Block
   func handleDownload(_ bgTask: WKURLSessionRefreshBackgroundTask) {
      let configuration = URLSessionConfiguration.background(withIdentifier: bgTask.sessionIdentifier)
      backgroundSession = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
      pendingBackgroundTasks.append(bgTask)
   }


In addition to the usual urlSession(_ session: , downloadTask: , didFinishDownloadingTo location: ) and urlSession(_ session: , task: , didCompleteWithError error: )I also implemented this below, though I haven't yet tested if this is really needed (I can see from the logs that it is called):

Code Block
   func urlSession(_ session: URLSession, task: URLSessionTask, willBeginDelayedRequest request: URLRequest, completionHandler: @escaping (URLSession.DelayedRequestDisposition, URLRequest?) -> Void) {
      completionHandler(URLSession.DelayedRequestDisposition.continueLoading, nil)
   }

Finally I mark the download done (the YourAppDataProvider has a var pendingBackgroundTasks = [WKURLSessionRefreshBackgroundTask]():

Code Block
   func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
      DispatchQueue.main.async {
         self.pendingBackgroundTasks.forEach {
            $0.setTaskCompletedWithSnapshot(true)
         }
         self.pendingBackgroundTasks.removeAll()
      }
   }


Finally, make sure the backgroundSession var is not a lazy var, but initiated in the YourAppDataProvider.init.

Now my new issue is that the complications do not update but show the complication templates...


Now my new issue is that the complications do not update but show the complication templates...

Got the complications working by
  1. Removing the app's complications from the clock faces

  2. Uninstalling the app from the watch

  3. Restarting the watch

  4. Reinstalling the app from the Xcode.



Hi Andy,
Very happy for you! So your first issue was because you were using SwiftUI, instead of a Storyboard? I'm glad you got it working. If you reboot your watch, does your app get started in the background and allowed to finish processing in the background?

Mine updates my complications well but only if I started the app, still has issues when launched by WatchOS.

If the watch was on the charger and locked when my app was woken up, I'd get a bunch of permission denied errors. This was due to various configuration files which I used that had the default complete file protection on, so my app couldn't access them if launched while the watch was locked. It still doesn't work in the most extreme case when on the charger, as I now get "Operation not permitted" in the  urlSession(_ session: , task: , didCompleteWithError error: ) method when trying to initiate the download.

Finally I noticed a lot of issues because of various data I had stored in the keychain accessible only if the user set a passcode and the device had to be unlocked when the app was launched. If you use the keychain you may want to review your code.

I test the app on 2 watches simultaneously, one running WatchOS 7 and the other running WatchOS 6.2.8. I don't know if it's because of WatchOS 7 or because that watch is my regular Apple Watch, as opposed to the other one which is just for testing, but my WatchOS 7 watch schedules tasks in a much more timely fashion.
Yes, using an Interface.storyboard and a HostingController instead of SwiftUI and @main App got the background ops working. Perhaps the App architecture is so in beta that it yet doesn't work, or I am missing something in my code to make App architecture solution work.

If I reboot the watch, and not launch my app, the background operation is executed after a while, looking at how the complication on the watch face changed. Then (without opening the app) I again changed the server JSON file and soon the complication was again updated. So no need to launch the app to get the background ops running. This was while the watch is on my wrist. I am requesting for the background task both in the Extension's applicationDidFinishLaunching() and applicationDidEnterBackground().

Reading your issues, I need to test more while watch is on the charger and/or locked. I am reading cached data from the watch using UserDefaults, so I have the previously fetched and stored data to show instead of "no data". And when the background fetch completes, then update the UserDefault storage and the complications.

ComplicationController does not request any network ops. Only thing done there is that the ComplicationController uses the MyAppDataProvider.shared (potentially initialising it?) to get the cached data and in the init, the background session configuration object URLSessionConfiguration.background and the session backgroundSession = URLSession are created. No network ops are explicitly requested from the ComplicationController. Don't know if this has any impact on anything. This is my first watchOS app and I am still a bit unsure when and how the OS instants and executes which methods under which process. Anyways, this may influence something since the session object - when created with the same session id - somehow creates (?) a connection with the NSURLSession daemon (?).
Hi Andy,

Regarding using SwiftUI for your app and the complications, maybe you missed something? In the WWDC'20 as you know there are videos demoing how to create complications using SwiftUI. Maybe that's only for >= WatchOS 7..

If your WatchOS app creates and writes to any files, the default is to have complete file protection, meaning those files will NOT be accessible for reading to your app if the screen is locked (regardless of whether it is on the charger or not).

So if your app is eventually started by WatchOS in the background, accessing any of those files will fail and throw an exception. The only files you can access when the screen is locked and your app is running in the background are either files you explicitly created with no file protection or the file(s) which WatchOS downloaded for you in the temp area via the background refresh download task.

When using background refresh either on WatchOS or iOS there's a tradeoff between usability and security. You most likely don't have to worry about that if the nature of your data doesn't require strong security, and from what you've shared, it looks like your app is fine.

My issues seem to stem from wanting to make a highly secure WatchOS app, but complications require security to be more lax than the default. I may have to change my app architecture and utilize complication push notifications instead of background refresh. We'll see.


OK so the use cases are so different that what I found out probably won't help you solve the issues your app has. Too bad. Hope you can solve the issue, if not by other means then by using push notifications.

I'll continue to work on the "SwiftUI way" at some point, maybe after the next beta comes out. Currently busy with an other app. Thanks for your comments Sal!
How often are the background URL session download tasks actually happening for you? I'm only getting one download every 1.5 hours, despite the docs and session videos saying we should expect up to 4 per hour.
@developer555, my app is getting them pretty much as expected. ExtensionDelegate launches scheduleBackgroundRefresh the first time 5 mins after, and later 20 mins later. After the background refresh is executed, the background URLSession is scheduled to happen after 5 mins. So the actual interval of URL download is 25 mins (first time it is 10 mins).
@AndyJJ would you mind posting your networking code as well as your background refresh code? Just trying to see if there's something I'm missing here. It's working fine on one of my test watches, but taking up to an hour on my personal watch.
WatchOS: background refresh file is downloaded but didFinishDownloadingTo: never called...
 
 
Q