"SKDownload encountered a sandboxing error" under iOS 14 only

Question

Created Aug ’20

Replies 23

Boosts 0

Views 4.6k

Participants 5

Hi there,

We're tracking an issue both in the sandbox environment and in production whereby download of App Store hosted in-app purchase content is not working properly. This is something that only happens on iOS 14 - all beta versions up to and including beta 3 - and has occurred with no change in our in-app purchase code. Either there is a bug in iOS 14 StoreKit, or something changed which breaks our implementation.

In production our logs show that in-app purchase SKPaymentTransaction object flow occurs nominally, with purchased and restored transactions presented and handled as expected.

Ultimately we provide content by calling

Code Block [[SKPaymentQueue defaultQueue] startDownloads:transaction.downloads]

with a non-nil transaction whose first downloads array object is non-nil and in the 'Waiting' SKDownload state. Ultimately a 'Finished' download object is returned via updatedDownloads:, but on processing the (supposedly) completed download NSFileManager fails to move the content files into the app directory.

Investigating in the Sandbox allows us to see more via the debugger.

It appears that the updatedDownloads: StoreKit method does not return 'Active' SKDownload objects at any time. We are starting a 'Waiting' SKDownload object and then seeing nothing until one of two cryptic iOS error message appears in the debugger:

SKDownload encountered a sandboxing error: 0

or

SKDownload encountered a sandboxing error: 35

The SKDownload is then returned to the updatedDownloads: method in the 'Finished' state, and not the 'Failed' state as one might have expected. When attempting to process the finished download we have verified via NSFileManager that the source files we expect DO exist at the source location (the temporary content download directory) and DO NOT exist at the destination location prior to attempting to move the files across. It would appear that the download is actually executing, but we are not being informed of its progress, and cannot move the resultant files into our app directory.

NSFileManager gives the following error on attempting a file move:

Error Domain=NSCocoaErrorDomain Code=513 "“redactedFilename.jpg” couldn’t be moved because you don’t have permission to access “redactedFolderName”." UserInfo={NSSourceFilePathErrorKey=/private/var/mobile/Library/OnDemandResources/AssetPacks/StoreKit/6050167547064785133/Contents/Content/25/redactedFilename.jpg, NSUserStringVariant=(
Move
), NSDestinationFilePath=/var/mobile/Containers/Data/Application/63151311-068D-4421-89BB-0E7AAEF65C62/Library/Application Support/Packs/redactedFolderName/redactedFilename.jpg, NSFilePath=/private/var/mobile/Library/OnDemandResources/AssetPacks/StoreKit/6050167547064785133/Contents/Content/25/redactedFilename.jpg, NSUnderlyingError=0x281aae2b0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}

We have double checked, and the target folder has been created and exists:

/var/mobile/Containers/Data/Application/63151311-068D-4421-89BB-0E7AAEF65C62/Library/Application Support/Packs/redactedFolderName

Can anyone shed any light on what is occurring here? Why do I not have permission to move files into the application support directory of my own app, into a folder which I created successfully moments prior? I've not found any hint of other people experiencing similar issues so far. If it is a simple case of needing to attain more permissions, that doesn't explain why I am not seeing 'Active' SKDownloads returned via StoreKit's updatedDownloads: method.

I would really appreciate any insight from anyone - thank you in advance.

Kind regards,

Alex

Boost

Answer 1

hithlimited OP

Aug ’20

As I can't edit my original post, here's a follow-up observation:

Under iOS 14, the SKDownload content appears to download to:

/private/var/mobile/Library/OnDemandResources/AssetPacks/StoreKit/…

and not to the app's /Library/Caches/ directory which has been the case on iOS 13 and prior.

The former directory is not in the app sandbox, and it makes sense we don't have permission to mess with it. If I change my code to copy instead of move items, everything works as expected again.

Why the change? Apple documentation (at developer. apple. com/documentation/storekit/in-apppurchase/unlockingpurchased_content?language=objc) states:

In iOS, your app can manage the downloaded files. The StoreKit framework saves these files for you in the Caches directory with the backup flag unset. After the download completes, your app is responsible for moving these files to the appropriate location. For content that can be deleted if the device runs out of disk space (and downloaded again later by your app), keep the files in the Caches directory. Otherwise, move the files to the Documents folder and set the flag to exclude them from user backups.

Therefore iOS 14 downloading to a different directory from which there is no permission to move files will break all apps that try to move files instead of copy them.

My guess is that the files are downloading there because of the SKDownload sandboxing error previously stated, and not because this is a wider change of behaviour under iOS 14. I couldn't see any mention of this anywhere in the release notes or documentation.

Can anyone official comment on whether this is an iOS 14 bug, a temporary scenario or whether this is set in stone from iOS 14 onwards?

1

Answer 2

codepoetz OP

Aug ’20

I have the same problems with iOS 14 beta 5. First, StoreKit downloads the files into OnDemandResources instead of into the documented Caches location. Then during the downloads, my app only receives updatedDownloads: events with SKDownloadStateWaiting (at the start) and then SKDownloadStateFinished (at the end). All of the usual SKDownloadStateActive events are missing so I can't update my progress bar during the download. Does anyone know what is going on with StoreKit in iOS 14?

1

Answer 3

futile OP

Aug ’20

I also have the exact same problems with iOS 14 beta 5. This is the first iOS 14 beta I've tried so I'm unable to say if it occurred in betas 1-4, but I can definitely say it did not occur in iOS 13.

Testing within the sandbox, I frequently see the following messages:

Code Block SKDownload encountered a sandboxing error: 0
SKDownload encountered a sandboxing error: 2

I'm unable to tell if these are benign or an early indication of the following problems:

Problem 1:

Store hosted content is now downloaded to /private/var/mobile/Library/OnDemandResources in iOS 14 (as reported by both codepoetz and hithlimited). This is easily solved by copying the content folder from that location rather than moving (as the app does not have permissions for the latter).

Problem 2:

During the download, paymentQueue:updatedDownloads: no longer receives SKDownloadStateActive events in iOS 14 and therefore it's not possible to indicate the download progress (as reported by codepoetz). This is a disconcerting experience for users who have just agreed to spend their money on IAP.

Currently I have no solution for problem 2.

0

Answer 4

hithlimited OP

Aug ’20

I'm glad to see this is not an issue isolated to our code. I filed feedback with Apple as of August 4th but have received no response as yet. I'll post here if and when I do.

0

Answer 5

futile OP

Sep ’20

Some extra info: I've just updated to iOS 14 beta 6 and there's no change in behavior since the beta 5. paymentQueue:updatedDownloads: is not called with SKDownloadStateActive events.

I'm going to work around this by adding a 'downloading' spinner to the app's UI, but I don't consider this a great UX for lengthy downloads - previously I had a download progress bar, which is a lot more reassuring for users.

0

Answer 6

hithlimited OP

Sep ’20

I've just updated to iPadOS 14 beta 7 and can confirm that both of these issues seem to have been fixed:

paymentQueue:updatedDownloads: is called with SKDownloads in SKDownloadStateActive
'Sandboxing' errors have disappeared and Apple hosted content is downloaded to the app ~/Library/Caches/Storekit/ directory

I haven't confirmed on iOS yet - can anyone else say? Let's hope it's fixed across the board and stays that way into production. I'm leaving defensive code in my app which will fall back to copying files if moving them does not work. Lack of download progress would still be a problem but at least content would install if these issues reared their ugly heads again.

0

Answer 7

futile OP

Sep ’20

I've just updated to iPadOS 14 beta 7 and my app's now unable to download store hosted content due to crashes in com.apple.NSXPCConnection.m-user.com.apple.storekitservice.

My app's IAP code hasn't changed since iOS 10, and I've never seen these crashes prior to iOS 14 beta 7. I've spent a day or so trying different things but no success so far.

Crash example 1:

Code Block Main Thread (1): "*** -[NSPlaceholderArray initWithObjects:count:]: attempt to insert nil object from objects[0]"
Enqueued from com.apple.NSXPCConnection.m-user.com.apple.storekitservice (Thread 9) Queue : com.apple.NSXPCConnection.m-user.com.apple.storekitservice (serial)
#0	0x000000010842a014 in dispatch_async ()
#1	0x00000001c0352ba0 in 34-[SKPaymentQueue downloadRemoved:]_block_invoke ()
#2	0x00000001a3b70038 in NSARRAY_IS_CALLING_OUT_TO_A_BLOCK ()
#3	0x00000001a3acfb90 in -[NSArrayM enumerateObjectsWithOptions:usingBlock:] ()
#4	0x00000001c0352a88 in -[SKPaymentQueue downloadRemoved:] ()
#5	0x00000001b9b7b35c in -[ASDStoreKitClientBroker downloadRemoved:] ()
#6	0x00000001a4fff554 in NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S1__ ()
#7	0x00000001a4e28828 in -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] ()
#8	0x00000001a4fff690 in message_handler ()
#9	0x00000001ea1fdbbc in _xpc_connection_call_event_handler ()
#10	0x00000001ea1fdf48 in _xpc_connection_mach_event ()
#11	0x00000001084256bc in _dispatch_client_callout4 ()
#12	0x00000001084411f8 in _dispatch_mach_msg_invoke ()
#13	0x000000010842cdd4 in _dispatch_lane_serial_drain ()
#14	0x0000000108442168 in _dispatch_mach_invoke ()
#15	0x000000010842cdd4 in _dispatch_lane_serial_drain ()
#16	0x000000010842dce8 in _dispatch_lane_invoke ()
#17	0x0000000108439e38 in _dispatch_workloop_worker_thread ()
#18	0x00000001ea1d8908 in _pthread_wqthread ()
#19	0x00000001ea1df77c in start_wqthread ()

Crash example 2:

Code Block Main Thread (1): "*** -[NSSetM removeObject:]: object cannot be nil"
Enqueued from com.apple.NSXPCConnection.m-user.com.apple.storekitservice (Thread 9) Queue : com.apple.NSXPCConnection.m-user.com.apple.storekitservice (serial)
#0	0x0000000105da6014 in dispatch_async ()
#1	0x00000001b449a854 in 40-[SKPaymentQueue downloadStatusChanged:]_block_invoke ()
#2	0x0000000197cb8038 in NSARRAY_IS_CALLING_OUT_TO_A_BLOCK ()
#3	0x0000000197c17b90 in -[NSArrayM enumerateObjectsWithOptions:usingBlock:] ()
#4	0x00000001b449a504 in -[SKPaymentQueue downloadStatusChanged:] ()
#5	0x00000001adcc31c0 in -[ASDStoreKitClientBroker downloadStatusChanged:] ()
#6	0x0000000199147554 in NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S1__ ()
#7	0x0000000198f70828 in -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] ()
#8	0x0000000199147690 in message_handler ()
#9	0x00000001de345bbc in _xpc_connection_call_event_handler ()
#10	0x00000001de345f48 in _xpc_connection_mach_event ()
#11	0x0000000105da16bc in _dispatch_client_callout4 ()
#12	0x0000000105dbd1f8 in _dispatch_mach_msg_invoke ()
#13	0x0000000105da8dd4 in _dispatch_lane_serial_drain ()
#14	0x0000000105dbe168 in _dispatch_mach_invoke ()
#15	0x0000000105da8dd4 in _dispatch_lane_serial_drain ()
#16	0x0000000105da9ce8 in _dispatch_lane_invoke ()
#17	0x0000000105db5e38 in _dispatch_workloop_worker_thread ()
#18	0x00000001de320908 in _pthread_wqthread ()
#19	0x00000001de32777c in start_wqthread ()

1

Answer 8

hithlimited OP

Sep ’20

We don't see the crashes you're reporting, but it does seem like there are still download issues.

Now users are intermittently seeing:

"mzafbenc.15091114977934116907" couldn't be moved to "StoreKit" because an item with the same name already exists. To save the file, either provide a different name, or move aside or delete the existing file, and try again.

This is not due to any file management on our part - it seems to be StoreKit failing to deal with the possibility that it already downloaded certain content and that it wasn't moved.

0

Answer 9

hithlimited OP

Sep ’20

We are also seeing -[NSURL initFileURLWithPath:]: nil string parameter exceptions when using content file paths from finished SKDownloads.

0

Answer 10

futile OP

Sep ’20

It seems that from the various changes in behavior recently that the iPadOS 14 StoreKit API is still a work in progress.

I could submit a bug report but in the past I've usually been required to provide sample code to demonstrate the problem and it will take days extract a simple example from a complex app. I think I'll play this by keeping my fingers crossed for a beta 8 and working on some other areas in the meantime.

I haven't yet tried this on the phone.

0

Answer 11

hithlimited OP

Sep ’20

Beta 8 is still showing these same problems - we can receive SKDownloads in the SKDownloadStateFinished state with a 'nil' contentURL property (prior to finishing the associated transactions).

I must say it's getting a bit uncomfortable now!

0

Answer 12

futile OP

Sep ’20

I also see no improvement in iPadOS 14 beta 8 (eg. see crash below, which occurs during store-hosted content download).

It's difficult to say how much we should worry without knowing the public launch date. I personally feel that these issues are so serious that it can't be released in this state. However, there are only 3 developers involved in this thread - I'd feel a lot more comfortable if there were more developers getting involved in the discussion.

Does this mean it works for everyone else?

Code Block Main Thread (1): "*** -[NSPlaceholderArray initWithObjects:count:]: attempt to insert nil object from objects[0]"
Enqueued from com.apple.NSXPCConnection.m-user.com.apple.storekitservice (Thread 2) Queue : com.apple.NSXPCConnection.m-user.com.apple.storekitservice (serial)
#0	0x0000000106556014 in dispatch_async ()
#1	0x00000001cc306ba0 in 34-[SKPaymentQueue downloadRemoved:]_block_invoke ()
#2	0x00000001afb24038 in NSARRAY_IS_CALLING_OUT_TO_A_BLOCK ()
#3	0x00000001afa83b90 in -[NSArrayM enumerateObjectsWithOptions:usingBlock:] ()
#4	0x00000001cc306a88 in -[SKPaymentQueue downloadRemoved:] ()
#5	0x00000001c5b2f35c in -[ASDStoreKitClientBroker downloadRemoved:] ()
#6	0x00000001b0fb3554 in NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S1 ()
#7	0x00000001b0ddc828 in -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] ()
#8	0x00000001b0fb3690 in message_handler ()
#9	0x00000001f61b1bbc in _xpc_connection_call_event_handler ()
#10	0x00000001f61b1f48 in _xpc_connection_mach_event ()
#11	0x00000001065516bc in _dispatch_client_callout4 ()
#12	0x000000010656d1f8 in _dispatch_mach_msg_invoke ()
#13	0x0000000106558dd4 in _dispatch_lane_serial_drain ()
#14	0x000000010656e168 in _dispatch_mach_invoke ()
#15	0x0000000106558dd4 in _dispatch_lane_serial_drain ()
#16	0x0000000106559ce8 in _dispatch_lane_invoke ()
#17	0x0000000106565e38 in _dispatch_workloop_worker_thread ()
#18	0x00000001f618c908 in _pthread_wqthread ()
#19	0x00000001f619377c in start_wqthread ()
Thread#0	0x00000001afba35ac in exceptionPreprocess ()
#1	0x00000001c3bad42c in objc_exception_throw ()
#2	0x00000001afc124cc in _CFThrowFormattedException ()
#3	0x00000001afc0fec4 in -[NSPlaceholderArray initWithObjects:count:].cold.3 ()
#4	0x00000001afa8c3b0 in -[NSPlaceholderArray initWithObjects:count:] ()
#5	0x00000001afa86ef0 in +[NSArray arrayWithObjects:count:] ()
#6	0x00000001cc306c1c in 34-[SKPaymentQueue downloadRemoved:]_block_invoke_2 ()
#7	0x000000010654fb68 in _dispatch_call_block_and_release ()
#8	0x00000001065515f0 in _dispatch_client_callout ()
#9	0x0000000106560890 in _dispatch_main_queue_callback_4CF ()
#10	0x00000001afb201e4 in CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE ()
#11	0x00000001afb1a3b4 in CFRunLoopRun ()
#12	0x00000001afb194bc in CFRunLoopRunSpecific ()
#13	0x00000001c651f820 in GSEventRunModal ()
#14	0x00000001b24bd8e0 in -[UIApplication _run] ()
#15	0x00000001b24c2fbc in UIApplicationMain ()
#16	0x00000001015d3d04 in main at /Users/mborstel/Documents/Projects/NS2/libs/Engine/Application/iOS/main.m:8
#17	0x00000001af7e0e60 in start ()

0

Answer 13

hithlimited OP

Sep ’20

I've just been doing some more analysis. I don't see the crash you're experiencing, but as of beta 7 and 8 I see:

Some finished SKDownloads work as expected - a valid contentURL in the current app caches/StoreKit directory is provided and the content can be moved from there.
Some finished downloads arrive with a nil download.contentURL and there is sometimes an associated 'StoreKit' alert implying that the downloaded files couldn't be moved to the caches 'StoreKit' directory (~Library/Caches/StoreKit/) because there are existing files that should be moved or deleted. It is StoreKit that is managing the download of these files - we are intended to install the files from this location later on but how can we do it if StoreKit doesn't give us a valid download.contentURL? It seems as if StoreKit should be responsible for clearing the download path in the event that content exists there.
Some finished downloads arrive with the wrong content path - an app UUID in the folder path which does not correspond to the current home directory - and no content in that directory (not even when replacing the home directory UUID with the correct one).

A real example of the latter is a download that was started fresh in a current app session, and reported a contentURL.path of:

/private/var/mobile/Containers/Data/Application/D135A96C-9991-4BB6-BF98-87B52CD2B25D/Library/Caches/StoreKit/3968122711419134656/Contents

when the current app home directory is different:

/var/mobile/Containers/Data/Application/E06F9149-D287-4D5A-BA08-6EA7DD469590/

No content can be found at the prior location, nor if we 'fix' the path by replacing the home directory UUID with the correct one for the current launch. You will see that kind of UUID mismatch for downloads started in previous app sessions, and actually updating the UUID will let you install the content if there is any, but in this case it just seems like the download finishes but there is no content whatsoever.

1

Answer 14

futile OP

Sep ’20

Out of interest, can I ask what your code does in response to SKDownloadStateWaiting?

I've always done this (recommended in a thread on StackOverflow to 'kickstart' a download which is stuck in a waiting state):

Code Block - (void)paymentQueue:(SKPaymentQueue *)queue updatedDownloads:(NSArray *)downloads
{
    for (SKDownload *download in downloads)
    {
		ASSERT(download != nil);
		
        switch (download.downloadState)
        {
			// ...
			// SNIP
			// ...
            case SKDownloadStateWaiting:
            	{
	                // Attempt to restart the download
	            	  [[SKPaymentQueue defaultQueue] startDownloads:[NSArray arrayWithObject:download]];
							}
              break;
					  default:
							break;
        }
    }
}

... however, I've found that this handler is called many times with SKDownloadStateWaiting, and therefore I call SKPaymentQueue:startDownloads: a huge number of times. I'll have to try some testing with this call disabled.

0

Answer 15

hithlimited OP

Sep ’20

Out of interest, can I ask what your code does in response to SKDownloadStateWaiting?

We do nothing for SKDownloadStateWaiting in the paymentQueue:updatedDownloads: method - in the switch we simply break for that case.

As I'm sure you do, we call [[SKPaymentQueue defaultQueue] startDownloads:] elsewhere based on the transactions we receive in paymentQueue:updatedTransactions:.

0