Level 1 (0 points)
-
Re: Background uploads with NSURLSession
Apple Staff (9,835 points)
Do you recommend me to use NSURLSessionUploadTask or another class?
NSURLSessionUploadTask
What do I have to do to continue all these uploads in background?
What do I have to do to continue all these uploads when app is not running?
Dump all the uploads into an NSURLSession background session.
For all of the first 3 points, do you recommend me to wrap NSURLSession objects in NSOperation ones in order to use an NSOperationQueue? Or iOS would use its own internal queue to properly handle NSURLSession objects?
That's hard to say. In general I'm a big fan of NSOperation but it's a tricky fit for NSURLSession background sessions because of the way that the background session interacts with your app's lifecycle. Specifically, a common sequence is this:
your app starts a bunch of background transfers
the user moves it into the background
the system suspends your app
the transfers continue in the background
the system terminates your app
the transfers continue in the background
the transfers complete
the system relaunches your app
If you're using an NSOperation to track each transfer, you have to reconstitute those operations when you're relaunched at step 8. Of course, you have to reconstitute something at step 8, it's just that reconstituting an NSOperation may be trickier than reconstituting some less complex object.
Share and Enjoy
—
Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"-
Re: Background uploads with NSURLSession
Level 1 (0 points)
Thanks Quinn!
I understand your point concerning NSOperationQueue. I guess iOS has its own queuing system for network requests...
But, are you saying that if I add IN ONLY ONE TIME 200 NSURLSessionUploadTask instances in the same NSURLSession configured with a background session configuration, iOS will propertly handle all of them?
If yes, is there any limit in term of number of tasks?
Thank you
Best
-
Re: Background uploads with NSURLSession
Apple Staff (9,835 points)
But, are you saying that if I add IN ONLY ONE TIME 200 NSURLSessionUploadTask instances in the same NSURLSession configured with a background session configuration, iOS will propertly handle all of them?
Yes. The NSURLSession background download system will happily deal with a few hundred requests. It serialises them internally, so only a few of those requests hit the 'wire' at any time.
If yes, is there any limit in term of number of tasks?
There is no hard limit but I generally recommend that you avoid pushing things too far. IMO hundreds of requests are fine, thousands of requests are pushing things, tens of thousands of requests would be silly.
I generally recommend that, if you have to deal with thousands of individual items, you zip them up and transfer them as one resumable transfer. There are, however, some gotcha there:
o Doing this sort of thing requires sophisticated server-side support.
o NSURLSession background sessions automatically handle resumable downloads (if the server supports it). That's not true for uploads. If you want to implement a resumable upload, you'll have to get involved each time the connection 'tears'. For that reason, it makes sense to chunk your uploads into reasonably sized chunks, so the upload get make a bunch of progress, even in the presence of connection failures, before your app has to resume again.
Share and Enjoy
—
Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"-
Re: Background uploads with NSURLSession
Level 1 (0 points)
Hey Quinn -- I've been reading a lot of your posts for the past hour trying to understand some gotcha's I'm facing with my app. Basically I'm trying to upload ~30 large files in the background, when the user's phone is plugged in and connected to wifi. I wrote a file manager that is dispatching these one-by-one, but from what I'm reading it looks like the better approach would be to send them all to NSURLSession at once.
Given this, I want to run a completion handler when all the files are finished uploading. I read your post about the resume rate limiter. Since the resume time backs off exponentially, I'm worried that by the time the 30th file finishes uploading, I won't be able to resume my app to handle the upload's completion handler. Since I have no clue which file is going to finish first, I can't pass nil as the completion handler for 29 and an actual completion handler for the 30th. Can you provide any insight?
-
Re: Background uploads with NSURLSession
Apple Staff (9,835 points)
In general an NSURLSession background session won’t resume (or relaunch) an app in the background until all of the requests in that session have completed. So, if you dump all 30 requests into a session and then get put in the background by the user, you’ll resume exactly once, when all 30 requests are complete.
When that happens you’ll see a sequence of events like this:
App resumes (or relaunches)
In the relaunch case you must remember to re-create your background session with the same identifier that you used originally. A common mistake is not re-creating the background session in the relaunch case. The best approach in most cases is to create that session in code called from
-application:willFinishLaunchingWithOptions:.System calls
-application:handleEventsForBackgroundURLSession:completionHandler:, passing it a completion handlerNSURLSession calls
-URLSession:task:didCompleteWithError:for each task that’s completeNSURLSession calls
-URLSessionDidFinishEventsForBackgroundURLSession:, at which point it’s appropriate to call the completion handler from step 2
IMPORTANT You can’t use NSURLSession’s convenience APIs, things like
-uploadTaskWithRequest:fromFile:completionHandler:, in a background session. You must use the delegate-based API, for example,-uploadTaskWithRequest:fromFile:. This makes sense when you think about it; if your app is terminated and then relaunched, there’s no way for the system to reconstitute the state held in a completion handler block.Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"-
Re: Background uploads with NSURLSession
Level 1 (0 points)
Thanks for the help. I'm currently using the S3 transfer utility library (https://cocoapods.org/pods/AWSS3) so I'll have to figure out what that's doing under the hood. I may need to write my own implementation based on signed URLs. Will report back when I find out.
-
Re: Background uploads with NSURLSession
Level 1 (0 points)
Hi,
I'm working on a similar iOS project, where I need to upload a very large video file (2 GB) to S3.
I tried setting up a single upload request to handle the entire thing, but as you pointed out, when there is a connection failure or when the user terminates the app, the upload transfer fails and I'd have to start over again.
If I were to split the 2 GB file into smaller 25 mb portions, I could add them all to my NSURLSession object to upload.
However, if the app is terminated and relaunched, and I retrieve the NSURLSession object, can I resume all of the pending upload requests? Does it have to upload all of the 25 mb files again or can it tell which ones have already been uploaded and only do the ones that haven't?
Thanks,
-Sreeni
-
Re: Background uploads with NSURLSession
Apple Staff (9,835 points)
However, if the app is terminated and relaunched, and I retrieve the NSURLSession object, can I resume all of the pending upload requests?
By “terminated”, do you mean:
“Removed from the multitasking UI”, so that all of your uploads get cancelled?
Terminated by the OS due to memory pressure, so that all of your uploads continue in the background?
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"-
Re: Background uploads with NSURLSession
Level 1 (0 points)
Hi,
By terminated, I mean “Removed from the multitasking UI”. Given that the app resumes (or relaunches) only when all tasks in the session complete, is there no way of keeping track of which files have been become uploaded and which are still pending? If all of the uploads become cancelled when a user force quits, then I'm guessing I can't retrieve the tasks that haven't completed (and were cancelled) and only resume those.
Is the best way to approach uploading a very large file to do a multipart upload with appropriate server side support so the upload can be resumed?
Thanks,
-Sreeni
-
Re: Background uploads with NSURLSession
Apple Staff (9,835 points)
Given that the app resumes (or relaunches) only when all tasks in the session complete …
To be clear, if the user removes your app from the multitasking UI then you won’t be resumed or relaunched in the background. The user will have to manually relaunch your app.
The behaviour for tasks that haven’t completed when the user removes the app from the multitasking UI varies by OS:
Originally the entire session would just disappear; it would be like you’d never issued those tasks
On current systems the tasks fail with
NSURLErrorCancelled
I believe the cutover was with iOS 8, that is, iOS 7 have the first behaviour and iOS 8 and later have the second, although I’ve never sat down to check that.
I’ve also never looked at the behaviour for tasks that have completed. Based on my understanding of how things fit together I suspect they’ll also get
NSURLErrorCancelled, but you’d have to test this.Regardless, the best way to recover from this situation is to talk to the server to see which of the uploads completed successfully. This is generally pretty easy to do via the
HEADHTTP method..Is the best way to approach uploading a very large file to do a multipart upload with appropriate server side support so the upload can be resumed?
That’s true for downloads but not for uploads. There’s actually a couple of problems doing a single, resumable upload:
With downloads, NSURLSession can automatically resume a failed download using HTTP standard technology (QA1761 gives an outline of how it works). For uploads there’s no way for NSURLSession to automatically restart the transfer, so it has to relaunch or resume your app to do the job. So, if you upload as a single file and the upload fails a bunch of times, you eventually provoke the ire of the .
There’s no way to tell NSURLSession to start a resume from some offset in a file. So if you have a large upload that fails, you have to make a copy of your upload file to remove the bytes at the front that have successfully been uploaded.
The alternative is to segment the file into relatively large chunks and upload those. This helps with both of these problems, at the cost of some code complexity.
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"-
Re: Background uploads with NSURLSession
Level 1 (0 points)
Thanks for the help! I think I know how to tackle the issue now.
-
Re: Background uploads with NSURLSession
Level 1 (0 points)
Hi,
I have two more questions, if you can offer any advice:
1. If I begin the transfers in the background (call the [uploadtask resume] while the app is in the background), and the discretionary property is automatically set to YES, what exactly does this mean? Does the phone have to be on wifi AND plugged in for it transfer or just be on wifi? What if I bring the app back to the foreground, is the discretionary property for these transfers set back to NO (given that session.discretionary = NO when I setup the background session)?
2. I setup and initiate around a 100 upload tasks to a NSURLSession object. If I lose network connectivity and regain it a while later (say 1 hour), does the session automatically resume all pending upload tasks? I'm only allowing uploads over wifi. I've been seeing the following strange behavior:
1. I initate 100 upload tasks in a single NSURLSession
2. 10 uploads were successfully completed
3. I lose wifi connection
4. I gain wifi connection an hour later
5. The session would complete another 15 upload tasks
6. The session stops uploading anymore data (the URLSession didSendBodyData is no longer being called, so I'm guessing all of the tasks got cancelled somehow).
Any help would be greatly appreciated,
Thanks,
-Sreeni
-
Re: Background uploads with NSURLSession
Apple Staff (9,835 points)
If I begin the transfers in the background (call the [uploadtask resume] while the app is in the background), and the discretionary property is automatically set to YES, what exactly does this mean?
To start, understand that the “discretionary property” isn’t an actual property, at least not one that you can see. Rather, it’s a value computer by NSURLSession based on a number of criteria, including:
The type of session (standard vs background)
The
discretionaryproperty of the configuration used to create that sessionWhether the app was in the foreground when the task was created
Whether the app is in the foreground right now
Moreover, the specific effects of this ‘property’ are not guaranteed; they have changed in the past and I fully expect them to change in the future.
Does the phone have to be on wifi AND plugged in for it transfer or just be on wifi?
In general, yes. However, it’s really up to the OS as to when it schedules discretionary tasks, and it’s possible that, for example, if the device ‘learns’ that it’s never on Wi-Fi, it may chose to run discretionary tasks over WWAN.
What if I bring the app back to the foreground, is the discretionary property for these transfers set back to NO (given that session.discretionary = NO when I setup the background session)?
Again, no property values change here but, yes, modern versions of iOS include the foreground state of the originating app when determining whether to run a discretionary request.
I setup and initiate around a 100 upload tasks to a NSURLSession object. If I lose network connectivity and regain it a while later (say 1 hour), does the session automatically resume all pending upload tasks?
Again, the exact behaviour is not specified and depends on a bunch of things. But, yes, I would expect these tasks to eventually complete.
(the URLSession didSendBodyData is no longer being called, so I'm guessing all of the tasks got cancelled somehow)
You shouldn’t need to guess here. If a task gets cancelled you should be told about it (via the standard task completion mechanism). You can also confirm whether a task is still around using
-getAllTasksWithCompletionHandler:.I expect what you’ll find is that the remaining tasks are still around, they’re just not running right now for some reason (lack of Wi-Fi, lack of power, resource budgets, and so on).
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardwarelet myEmail = "eskimo" + "1" + "@apple.com"
-
-
-
-
-
-