Why does CloudKit data field (bytes) take up more (~10 times) iCloud storage than asset field?

While CKAssets have a little more overhead, this isn't expected. please file a bug report. Thanks.

The bug report guy (Feedback Assistant) reply that it “behaves as intended”. I reply back with some ramifications of the behavior.

Is there anything else I can do?
Is there anything else you, Frameworks Engineer , can do? 
Can you , talk to each other? 
How are things working here?

The bug report guy (Feedback Assistant) reply that it “behaves as
intended”.

What was your bug number?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

Feedback ID:
FB8644159

This is what I think is the easiest way to replicate the problem:

Requirements: 

1. an iPhone and a laptop (with a web browser) both connected to the internet. 

2. An apple developer account. The iPhone is signed in with the same Apple ID.

3. An existing CloudKit container’s private database that has a record with data in Bytes field.

4. Make sure your other personal iCloud data usage remain the same throughout the test.

Step 1:

Go to Your iPhone > Settings > Apple ID > iCloud > Manage Storage
Find the currently used space in bytes. Write it down. Let’s say the number is “A”.  

Step 2:

Via you laptop’s browser and the developer account, go to CloudKit Dashboard > Container > Data
Query a record that has a “Bytes” field. Click the record name to open “Edit Record” window.
The field shows the space (in bytes) that the data occupy. Write it down. Let’s say the number is “B”.  

Step 3:

Delete the data in that ”Bytes” field by clicking at the rounded-red button on the right. Then save the record.

Step 4:

Wait a few seconds for iCloud to sync.
Go back to Your iPhone > Settings > Apple ID > iCloud > Manage Storage
Find the currently used space in bytes (same place in step 1). Write it down. Let’s say the number is “C”.  

Step 5:

To recap,
A is the used space BEFORE deleting data,
B is the size of the data deleted from the Bytes field,
C is the used space AFTER deleting data.

The parity should be: A - C = B
But what actually happen is: A - C = 10*B

I did it many times. Same result.

@DD21 Did you end up finding a solution? I'm using NSPersistentCloudKitContainer to have CloudKit + Core Data and I'm observing the same issue. I have saved ~6000K items in Core Data, which some of them have images. The local size for the sum of the images is around 76 MB, but if you check it on iPhone > Settings > Apple ID > iCloud > Manage Storage, it shows up as ~760 MB, which is the 10x increase in size that you reported a year ago.

This seems to be a bug on the whatever's in charge of calculating the size for iCloud assets, because when I delete my app from my device and let it download all the information stored on iCloud again, it finishes the download extremely fast (I'm assuming because it's only downloading 76MB). If it were downloading 760MB, it would take much longer knowing my network connection speed.

Now, if this bug actually affects user's iCloud quota (which it seems that it does), it is a large scale issue that might be impacting millions of users. Users might have much more iCloud storage space than they think because of this. cc // @eskimo

This is insane. I've been investigating this issue and I can reproduce it 100% of the time on my end. I have two methods in my code that I've added to debug this issue, one to calculate the size of the thumbnails and another one to delete them. Here are both of them:

func allThumbnailsSize() -> String {
    let items = (try? self.container.viewContext.fetch(CDItem.fetchRequest())) ?? []
    let size = items.reduce(0, { $0 + ($1.metadataThumbnail?.count ?? 0) })
    let formatter = ByteCountFormatter()
    formatter.formattingContext = .middleOfSentence
    return formatter.string(fromByteCount: Int64(size))
}

func deleteAllThumbnails() {
    let items = (try? self.container.viewContext.fetch(CDItem.fetchRequest())) ?? []
    items.forEach { $0.metadataThumbnail = nil }
    self.saveIfNeeded()
}

And here's the property in the model:

Here's a screenshot that clearly shows the problem, and how simply nullifying the property that is of type Binary Data in Core Data reduces the size ~10x on iCloud.

1. Capture of the size in the settings of my app. The size is calculated using allThumbnailsSize() shown above.
2. Capture of the size reported on iCloud for Abyss, my app. (Ignore the other Abyss record with 80.9MB, it's another iCloud Container for a migration)
3. Capture of the size after tapping the Delete Thumbnails button. That button simply calls deleteAllThumbnails().
4. Capture of the size reported on iCloud for Abyss after a few minutes to let it sync.

As shown on the images, the iCloud size is totally wrong. And it's not an issue on the iPhone only, checking the size on iCloud on my Mac, also reported 782.2 MB used.

For apple folks, I've opened a feedback: FB9770168

I've been digging deeper on this issue because it's affecting my app, and the problem is clearly and issue calculating the size on iCloud. The information that I'm adding now it's already on the FB9770168, but I'm posting it anyway here so non-Apple employees can see the findings.

1. Size reported on my app's settings by calling allThumbnailsSize() shown above.
2. Size reported for iCloud Storage
3. Size reported by iPhone Storage (from the 216.6 MB, you'd have to subtract ~100MB from another database that I have on the app.)

iCloud size reported for the same app on my Mac:

As you can see, even if the 216.6 MB were true (which they are not, explained above), the iCloud size it totally wrong.

I even went a step further and decided to perform a test to see how much data was being downloaded from iCloud. What I did is delete the app from my device, reset the network statistics, let it download all the information from iCloud, and check the statistics again.

As shown on the images, if the iCloud size was correct, the download of all my data should have consumed ~757.1 MB. What it actually consumed is just ~78 MB, which is basically the size of the images reported in my app's settings + metadata of the object (small strings + bools)

Has there been any progress with this issue?

I'm experimenting with an app that uses CloudKit to connect a user's devices and I'm seeing surprisingly high iCloud storage usage when checking in Settings on the device.

This makes me wonder if I have a hidden bug that is leaking data/records to iCloud. Debugging this feels a bit cumbersome because it involves many record types and zones.