We're trying to implement a backup/restore data feature in our business productivity iPad app using UIDocumentPickerViewController and AppleArchive, but discovered AppleArchive crashes instead of failing gracefully when decrypting a corrupt archive.
As described in forum post 765101, UIDocumentPickerViewController can handoff a corrupt copy of an archive to UIDocumentPickerDelegate under specific circumstances.
We've duplicated this behavior with iPadOS 16.6.1 and 17.7 when building our app with Xcode 15.4 targeting minimum deployment of iPadOS 16. We haven't tested this with the bleeding edge iPadOS 18.
Our app is primarily Objective-C, but it utilizes the Swift-based AppleArchive 'EncryptingAndDecryptingDirectories' sample code associated with WWDC21 session:
10233: Bring Encrypted Archives and Performance Improvements to Your App with Accelerate.
The WWDC21 'EncryptingAndDecryptingDirectories' Swift sample project crashes in a similar manner when a corrupt archive file created by UIDocumentPickerViewController is dropped into the sample app's window for decryption (see attached crash log).
Does anyone know if there's a workaround for the 'EncryptingAndDecryptingDirectories' sample project to prevent AppleArchive from crashing when decrypting a corrupt archive?
crash log.txt
Apple Archive
RSS for tagPerform multithreaded lossless compression of directories, files, and data.
Posts under Apple Archive tag
6 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
The AppleArchive module is pretty cool, but it relies almost entirely on two stream types, ArchiveByteStream and ArchiveStream, which don't really work well in a Swift Concurrency-based workflow since they all use thread-blocking mechanisms like pthread mutexes for synchronization, and threads in the cooperative pool should not be blocked. They also use their own thread pools for processing, independently of the cooperative thread pool, making it easy to end up with more threads than one has cores. (Perhaps even more so if you try to compress/decompress multiple files at once? The documentation isn't clear on whether separate archive operations share the same thread pool or not, but since it allows you to choose the size of the pool, it seems that these may be separate from each other as well.)
Are there any plans for an interface to Apple Archive that would fit better with the structured concurrency model?
I've implemented Apple Archive in the app I'm working on and have implemented lzfse algorithm. The framework is extremely impressive in terms of the speed and compression ratios. I'm using the closure in ArchiveStream.writeDirectoryContents to indicate what the current file is being processed, but there doesn't seem to be a way to pull out progress % on each file. I've seen examples here [Accelerate].(https://developer.apple.com/documentation/accelerate/compressing_and_decompressing_files_with_stream_compression) but these aren't using the Apple Archive Framework... I'd really like to stick with the Apple Archive Framework if possible as it's simple and very fast.
I very much love the performance of AppleArchive and how approachable it is, and believe it to be one of the most underrated frameworks in the SDK. In a scenario quite typical, I need to compress files and submit them to a back end, where the server handling the files is not an Apple platform. Obviously, individual files compressed with AA will not be compatible with other systems out of the box, but there are compatible compression algorithms.
ZLIB is recommended for cases where cross-platform compatibility is necessary. As I understand it, AA adds additional headers to files in order to support preservation of file attributes, ownership and other data. Following the steps outlined in the docs, I've written code to compress single files. I can easily compress and decompress using AA without issue.
To create a proof-of-concept, I've written some code in python using its zlib module. In order to get to the compressed data, it's necessary to handle the AA header fields. The first 64 bytes of a compressed file appear as follows:
AA documentation states that ZLIB Level 5 compression is used, and comes in the form of raw DEFLATE data prefixed with two header bytes. In this case, these bytes are 78 5e, which begin at the 28th byte and appear as x^ above. My hope was that seeking to the start of the compressed data, then passing what remains to a decompressor object initialized with the correct WBITS would work.
It works fantastically for files 1MB or less in size. Files which are larger only decompress the first megabyte. The decompressor object is reaching EOF, and I've tried various ways of attempting to seek to and concatenate the other blocks, but to no avail.
Using the older Compression framework and the method specified here, with the same algorithm, yields different results. I can decompress files of any size using python's zlib module. My assumption is that AppleArchive is doing something differently in order to support its multithreading capabilities, perhaps even with asymmetric encoding where the blocks are not ordered.
Is there a solution to this problem? If not, why would one ever use ZLIB versus the much more efficient LZFSE? I could use the older Compression API, but it is significantly slower compressing synchronously, and performance is critical with the application I am adding this feature to.
VS 2022 .net MAUI Error : Process cannot be executed on XMA server. There was an error unzipping the file : The given path's format is not supported. Please Help!
I support Mac code signing and notarisation for DTS and, as part of that work, I often need to look inside various Apple-specific archive file formats. This post explains how I do this. It’s mostly for the benefit of Future Quinn™, but I figured other folks would appreciate it as well.
IMPORTANT This post explains low-level techniques for inspecting archives. Do not use them to create archives. Instead, create your archives using the highest-level tool that will get the job done [1].
Flat Installer Package
A flat installer package — appropriate for uploading to the Mac App Store or the notary service — is actually a xar archive. Unpack it using the xar tool. For example:
% # List the contents:
%
% xar -tf InstallTest-1.0d1.pkg
com.example.apple-samplecode.InstallTest.pkg
com.example.apple-samplecode.InstallTest.pkg/Bom
com.example.apple-samplecode.InstallTest.pkg/Payload
com.example.apple-samplecode.InstallTest.pkg/PackageInfo
Distribution
%
% # Actually unpack:
#
% mkdir tmp
% cd tmp
% xar -xf ../InstallTest-1.0d1.pkg
% find .
.
./Distribution
./com.example.apple-samplecode.InstallTest.pkg
./com.example.apple-samplecode.InstallTest.pkg/Bom
./com.example.apple-samplecode.InstallTest.pkg/Payload
./com.example.apple-samplecode.InstallTest.pkg/PackageInfo
See the xar man page for more info on that tool.
The resulting Bom file is a ‘bill of materials’. For more on this, see the bom man page for details. Use lsbom to dump this:
% lsbom ./com.example.apple-samplecode.InstallTest.pkg/Bom
. 0 0/0
./InstallTest.app …
./InstallTest.app/Contents …
./InstallTest.app/Contents/Info.plist …
./InstallTest.app/Contents/MacOS …
./InstallTest.app/Contents/MacOS/InstallTest …
…
The Payload file contains… you guessed it… the installer’s payload. This is a gzipped cpio archive. To unpack it, pipe the file through cpio:
% cpio -i < com.example.apple-samplecode.InstallTest.pkg/Payload
5072 blocks
% find InstallTest.app
InstallTest.app
InstallTest.app/Contents
InstallTest.app/Contents/Info.plist
InstallTest.app/Contents/MacOS
InstallTest.app/Contents/MacOS/InstallTest
…
See the cpio man page for more info on that tool.
Note This is a bit of a hassle so most of the time I use a third-party app to unpack installer packages. Which one? Well, I can’t give away all my secrets (-:
Xip Archives
To extract a xip archive (pronounced, I believe, as chip archive), run the xip tool with the --expand argument:
% xip --expand XipTest.xip
However, if that doesn’t work you’ll need to dig into the archive. First, undo the outer xar wrapper:
% xar -xf XipTest.xip
This produces two files, Content and Metadata:
% ls -l
total 7552
-rw-r--r-- 1 quinn staff 1683391 10 Jun 17:05 Content
-rw-r--r-- 1 quinn staff 287 10 Jun 17:08 Metadata
-rw-r--r-- 1 quinn staff 1697157 10 Jun 17:05 XipTest.xip
The Metadata file is an XML property list:
% cat Metadata
…
<dict>
<key>UncompressedSize</key>
<integer>2598653</integer>
<key>Version</key>
<integer>1</integer>
</dict>
</plist>
The Content file is an Apple Archive. Unpack this using the aa tool:
% aa extract -ignore-eperm -i Content -d tmp
% find tmp
tmp
tmp/XipTest
tmp/XipTest/XipTest.app
tmp/XipTest/XipTest.app/Contents
tmp/XipTest/XipTest.app/Contents/Info.plist
tmp/XipTest/XipTest.app/Contents/MacOS
tmp/XipTest/XipTest.app/Contents/MacOS/QCodeIndex
tmp/XipTest/XipTest.app/Contents/MacOS/XipTest
…
See the aa man page for more info on that tool.
Note aa was previously known as yaa.
iOS App Archives
iOS apps are stored in an .ipa file. This is actually a zip archive under the covers. To unpack it, change the file name extension to .zip and then double click it it in the Finder (or use your favourite unzipping tool, like unzip or ditto).
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
[1] For installer package specifically, productbuild is your friend, but you can also use the lower-level tools like productsign, pkgbuild, and pkgutil.
Revision History
2024-02-20 Added the iOS App Archives section. Added a note about third-party apps to the end of the Flat Installer Package section.
2022-09-30 Changed yaa to aa and added a reference to the Apple Archive framework.
2021-02-26 Fixed the formatting.
2020-06-10 First posted.