We're experiencing an extremely bizarre crash in our iOS/iPadOS app.
Some major changes in our latest release:
- Built with Xcode 13 on Big Sur 11.6
- Added Hardened Runtime entitlement
- Enabled Hardened Runtime in build settings
- Dropped support for iOS 13
- Updated Firebase to 8.8.0
- Built using the latest fastlane version
OK here's the bizarre part: our app works fine for users who download and install the app directly from the App Store, on both iOS 14 & 15.
However, iOS 14 users who get the app pushed to their phone by MDM and install it from a dialog box like this will have it crash on launch:
So what is the difference between launching the app after installing it yourself, and launching it after MDM installs it—in both cases, the app originates from the App Store?
Here is an example crash log:
{"app_name":"REDACTED","timestamp":"REDACTED","app_version":"REDACTED","slice_uuid":"REDACTED","adam_id":REDACTED,"build_version":"REDACTED","platform":0,"bundleID":"com.REDACTED.REDACTEDapp","share_with_app_devs":0,"is_first_party":0,"bug_type":"109","os_version":"iPhone OS 14.6 (18F72)","incident_id":"REDACTED","name":"REDACTED"}
Incident Identifier: REDACTED
CrashReporter Key: REDACTED
Hardware Model: iPhone11,8
Process: REDACTED [1561]
Path: /private/var/containers/Bundle/Application/REDACTED/REDACTED.app/REDACTED
Identifier: com.REDACTED.REDACTEDapp
Version: REDACTED
AppStoreTools: 13A1030d
Code Type: ARM-64 (Native)
Role: Foreground
Parent Process: launchd [1]
Coalition: com.REDACTED.REDACTEDapp [562]
Date/Time: 2021-10-26 15:17:43.5707 -0500
Launch Time: 2021-10-26 15:17:43.1390 -0500
OS Version: iPhone OS 14.6 (18F72)
Release Type: User
Baseband Version: 3.04.01
Report Version: 104
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000115596b28
Termination Signal: Segmentation fault: 11
Termination Reason: Namespace SIGNAL, Code 0xb
Terminating Process: exc handler [1561]
Highlighted by Thread: 0
Backtrace not available
Unknown thread crashed with ARM Thread State (64-bit):
x0: 0x000000016d11d5c0 x1: 0x0000000000000008 x2: 0x0000000000000000 x3: 0x0000000000000000
x4: 0x000000016d11d590 x5: 0x0000000000000020 x6: 0x736665727373616c x7: 0x0000000000000600
x8: 0x000000005f24901f x9: 0x000000000f8884a0 x10: 0x000000000000901c x11: 0x0000000000000003
x12: 0x0000000000ff0006 x13: 0x000000000000c000 x14: 0x0000000000000001 x15: 0xffffffffffffffff
x16: 0x00000001030879cc x17: 0x6ae100016d11d440 x18: 0x0000000000000000 x19: 0x0000000115596b28
x20: 0x000000016d11daf8 x21: 0x000000016d11dad0 x22: 0x0000000105d00000 x23: 0x000000000f896b28
x24: 0x000000005f24901f x25: 0x0000000105d0e680 x26: 0x000000000000e688 x27: 0x000000016d11db08
x28: 0x0000000000001b99 fp: 0x000000016d11d6a0 lr: 0xba6bf2010306bf84
sp: 0x000000016d11d620 pc: 0x000000010306beec cpsr: 0x00000000
esr: 0x00000000 Address size fault
Binary images description not available
Error Formulating Crash Report:
Failed to create CSSymbolicatorRef - corpse still valid ¯\_(ツ)_/¯
EOF
We don't see these crashes appearing in Crashlytics. I'm not sure if they're appearing in Xcode organizer, but if they are I haven't been able to find them.
I can reproduce the crash locally using our UEM test account for VMWare Workspace One (AirWatch), but it does not happen on AdHoc or Developer builds, nor does it happen if Xcode is connected with a live debugger session.
We tried regenerating all our provisioning profiles with DER encoding, then re-signing and re-submitting the app to Apple, but it did not help the problem.
Any tips would be most appreciated as we are extremely befuddled by what could possibly be causing this.
-
—
O_G
-
—
O_G
-
—
O_G
Add a CommentUpdate:
After turning on App Analytics, this crash still isn't syncing up to Xcode's backend. That makes me think it's a watchdog process termination.
As well, the .000.ips info of the crash adds the following:
_dyld_process_info_create failed with 6What does this mean?
Why would this happen only when the app is installed via MDM, and only on iOS 14?
Upon further investigation I noticed a Jetsam event around this crash implicating the app's process in being terminated due to using too much memory (2GB!) (see report below). Using up all the memory could only be possible if we have an infinite loop, and I also found several similar crash reports, with EXC_BAD_ACCESS and KERN_INVALID_ADDRESS or KERN_PROTECTION_FAILURE with Segmentation Fault 11 and Code 0xb, that indicate this could be caused by an infinite loop.
So, what could cause an infinite loop, but only for MDM installs? I found another thread on this forum (link here) where someone had incurred an infinite loop due to following this code example from WWDC 2013 (link here) where an MDM manager class sets up an observer on NSUserDefaultsDidChangeNotification. Their infinite loop was due to someone putting a write call to UserDefaults in the target method of that observer. >.<
So I checked and, sure enough, we have a write call to UserDefaults in the target method of the observer on NSUserDefaultsDidChangeNotification in our MDM manager. It was wrapped with an ivar to prevent the recursion, but it was not a perfect solution obviously. Something about how our iOS 14 build is optimized or the timing of when the method gets called on those builds triggers the recursion, which doesn't happen in iOS 15.
In any event... the fix would be to not write to UserDefaults in that method if the change we want to make in UserDefaults already exists. So we're trying that, will reply here if it worked.
{ "uuid" : "1b20964b-84fa-329a-b8d0-7ad5df6f42ce", "states" : [ "frontmost" ], "killDelta" : 5005, "genCount" : 0, "age" : 73179629, "purgeable" : 0, "fds" : 50, "coalition" : 1975, "rpages" : 196608, "priority" : 10, "reason" : "per-process-limit", "physicalPages" : { "internal" : [ 135870, 60638 ] }, "pid" : 21668, "cpuTime" : 1.3859840000000001, "name" : "REDACTED", "lifetimeMax" : 196608 },Nevermind, it has nothing to do with the hypothesized infinite loop that I mentioned in the prior post. We ruled it out by disabling our MDM code on first launch and adding a bunch of os_log calls at the start of the app. None of the os_log calls get called, meaning AppDelegate never gets initialized. So we're back to square one trying to figure out this mystery.