Can CryptoKit encrypt or decrypt streams?

My question is: Can CryptoKit decrypt or encrypt a stream without a
complete read?

Hi,

I created an enhancement request number: FB11471366

I’ve been discussing your bug report (FB11471366) with the CryptoKit folks and I wanted to pass along a potential workaround, and some info about that workaround’s limitations.

The workaround is trivial: To use AES.GCM with a large data set, write it to a file, memory map the file using Data(contentsOf:options:) using one of the ‘mapped’ options, and then pass that to CryptoKit.

However, this workaround has limitations. Specifically:

• For a very large file you may run out of address space. This is most noticeable on iOS, where the default address space is rather limited.

• This only works if the file is on a ‘safe’ volume, that is, a volume that can’t fail [1]. On iOS this means the volume holding your app’s container. On macOS this means the root volume, the volume holding the user’s home directory, and the volume holding your app.

  Memory mapping files on other volumes is not safe. If the volume goes away for some reason — for example, it’s on a USB drive and the user unplugs that drive — the resulting read errors are surfaced to your process as machine exceptions [2]. There is no reasonable way to catch such exceptions, so your only option is to avoid memory-mapped files on such volumes.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Well, all volumes can fail so this should be “a volume where any failure is going to cause bigger problems for your app”.

[2] For a definition of that term, see What is an exception?.

Hi, 

We do have the same problem on the app we're working on. Today we are using CommonCrypto for cryptography and we plan to switch to CryptoKit (since it's newer), but we can't until it will support incremental steps for encryption/decryption. In case you have big data (e.g. 2-3 GB) it's impossible to use "1 shot encryption/decryption" since the app will load up too much data in RAM, which will then kill the app (with a memory warning). Having only seal/open APIs won't help with large files cryptography.

Thanks, Alin

I recommend that you file your own bug about this, making sure to:

• Reference the bug number above.

• List the specific algorithms you need.

• Describe your big data requirements.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

This came up again in a different context and that reminded me of something: I recently learnt why combining GCM and streaming is problematic.

IMPORTANT I’m not an expert on secure design; please keep that in mind as you read the following.

One general rule of security is that you shouldn’t do encryption without authentication. GCM is an authenticated encryption algorithm, which is cool. However, the authentication doesn’t actually happen until you get to the end of the data. This isn’t a problem for one-shot APIs, because they do the decryption and authentication in one go. However, it’s a serious problem for streaming APIs. These give you back chunks of decrypted text before the authentication is done and, in general, it’s not safe to work with such data.

If you want to learn more about this, hopefully from someone who is a secure design expert, search the ’net for the term releasing unverified plaintext, or RUP for short.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@eskimo That makes sense; in particular, if I'm decrypting content and streaming it out over the network as I go, I may end up streaming out invalid information, with no way to "take it back" once I get to the authentication tag and realize that it's invalid. That is certainly a case where streaming could go wrong.

In my case, I'm just trying to encrypt a large video file on disk. It may be as large as 300MB. Currently, we're using OpenSSL to read a chunk, encrypt it, and write it to a local file on disk. If authentication fails at the end, then we just delete the entire file; nobody's going to be working with this intermediate data. We do this iteratively so that I don't end up with 300 MB of plaintext and 300 MB of ciphertext both in memory at the same time; that's likely to kill my app. I could use memory mapped Data to avoid loading the full plaintext input into memory, but with the current APIs I don't see any way to avoid having 300 MB of ciphertext given back to me.

Perhaps an API that could take a file URL to write the complete output to would be sufficient?