How to find freeze that doesn't happen to me?

A handful of my customers have reported a consistent freeze in my app (smells like either a thread deadlock or an infinite recursion to me). It's working properly for most people, but if 3 people are reporting it to me that says to me that probably 300 are experiencing it but staying quiet.

It happens every time they take a certain kind of action. But it never happens to me so I can't figure out what is breaking.

No crash is happening, so I'm not getting any reports via Firebase.

It doesn't appear to be related to their data, because they can have someone else use a different device, login to their account, download all of their data, and then successfully do the thing that freezes 100% of the time on their original device.

I've got one of these customers set up on TestFlight so I can send them test versions of the app with some debug collection code. I created my own version of a tracelog and write a line to a file whenever I start/end a function. I've added those tracelog calls to a bunch of methods and classes (but not all).

The action that causes the freeze is when they tap Edit on a record. What's supposed to happen is that I modally present a nav controller containing an EditBlahViewController.

The trace logs show that the EditBlahViewController gets through viewDidLoad, viewWillAppear, two rounds of viewWillLayoutSubviews/viewDidLayoutSubviews... but then never gets to viewDidAppear.

I've tried updating more and more functions/classes with the tracelog calls, but I can't find any infinite recursions. I've added tracelogs to all calls involving dispatching to other threads (especially synchronous dispatches to the main thread) but I don't see any deadlocks. Every dispatch to main starts and ends properly and then goes on to the next thing.

User is on iOS 16.6.1 with an iPhone 12.

I don't know what else to try. How can I debug this at a distance and figure out where the problem is?

Answered by kennywyland in 765942022

This might be a bug in iOS 16/17 (all the users reporting the problem were on iOS 16.x).

When I was running iOS 16 on my sim and using the iOS 16-compatible version of Xcode, the freeze was never happening to me. iOS 17 got released along with a new version of Xcode, so I updated my Xcode to the new version (I was using the latest one before this as well). Once I did that, I started getting the UI to freeze up for me in the simulator! Woo!

Now that it was happening to me, I was able to run Instruments and track the stacktrace where the UI was locking up more precisely. I got a slightly different stacktrace which referenced a UICalendarView before the UICollectionView stack frames. (I forgot to capture a copy to paste here.) I wasn't familiar with UICalendarView but it appears to be a new replacement for UIDatePicker.

I finally found that viewWillLayoutSubviews was being called over and over, forever. I wasn't changing the size of anything in my UI during those calls... but deep in the iOS code, it was. Something in the UICalendarView code was making some kind of layout change every time layoutSubviews was called, so the system would call viewWillLayoutSubviews again and again, forever because something kept changing.

My UIDatePicker would start off-screen in my storyboard layout and then when it was needed, I would change the constraints in my layout to be smaller and that would cause the UIDatePicker to slide onto the screen.

The only workaround that I found to avoid this weird iOS bug, was to remove the UIDatePicker from my storyboard and then create one in code instead. And instead of doing it during initial creation of the view, I deferred the creation until just before I was going to show the UIDatePicker. And then whatever tickled the weird bug in UICalendarView just didn't happen.

One possible debugging approach, assuming you have the ability to communicate with the user who experience the issue, is to ask them to take a spindump using Activity Monitor when the behavior occurs. That should help you decide whether this is a deadlock, a loop, or something else.

The trace logs show that the EditBlahViewController gets through viewDidLoad, viewWillAppear, two rounds of viewWillLayoutSubviews/viewDidLayoutSubviews... but then never gets to viewDidAppear.

If the code isn't looping forever, then my wild guess at the cause of this sort of behavior is that the view controller is getting deallocated, which would lead to an apparent "freeze" because that piece of the UI just isn't happening. :)

I added dealloc methods to the UIViewController subclass and UINavigationController subclass that I was presenting. I put my tracelog calls in those deallocs and they were never called. So, the view controllers aren't getting dealloc'd between viewWillAppear and viewDidAppear.

Hi @kennywyland. I ran into that issue yesterday. The approach I have found is to integrate main thread's watcher (examples of how to implement - https://github.com/tiket-apps-ci/Watchdog). And call fatalError in case main thread does not respond more than a threshold value. So, eventually it is supposed to be logged in firebase along with a stack trace.

Thanks @daniil_berlin, that's a great tool! I used it for this and I'll definitely be using it in the future. The sad part is that when I got the Firebase crash report for the Watchdog, literally every thread is running in Apple code, not my code. I can't see anything that would be causing my freeze/deadlock. The main thread seems to be doing some UICollectionView code but I don't even -have- a UICollectionView in my app.

Any idea where this could be or what is happening?

Here is what it shows:


com.apple.main-thread
0  libobjc.A.dylib                0xa6d0 objc_alloc_init + 36
1  UIKitCore                      0x14e7f8 +[UICollectionViewLayoutAttributes layoutAttributesForCellWithIndexPath:] + 24
2  UIKitCore                      0x14e0a0 -[_UICollectionCompositionalLayoutSolver _createAndCacheLayoutAttributesForItemAtIndexPath:frame:zIndex:] + 152
3  UIKitCore                      0x13a04 __85-[_UICollectionCompositionalLayoutSolver _layoutAttributesForElementsInRect:handler:]_block_invoke + 944
4  UIKitCore                      0x6b799c __85-[_UICollectionCompositionalLayoutSolver _layoutAttributesForElementsInRect:handler:]_block_invoke_2 + 40
5  UIKitCore                      0x484278 _UIRTreeContainerNode::enumerateIndexesIntersecting(CGRect const&, bool*, void (unsigned long, bool*) block_pointer) const + 160
6  UIKitCore                      0x4842b4 _UIRTreeContainerNode::enumerateIndexesIntersecting(CGRect const&, bool*, void (unsigned long, bool*) block_pointer) const + 220
7  UIKitCore                      0x40adb0 -[_UIRTree enumerateIndexesForFramesIntersectingFrame:withBlock:] + 52
8  UIKitCore                      0x11b14 -[_UICollectionCompositionalLayoutSolver _layoutAttributesForElementsInRect:handler:] + 1092
9  UIKitCore                      0x4483a4 -[_UICollectionCompositionalLayoutSolver layoutAttributesForElementsInRect:] + 172
10 UIKitCore                      0x1321c -[UICollectionViewCompositionalLayout layoutAttributesForElementsInRect:] + 112
11 UIKitCore                      0x12840 __45-[UICollectionViewData validateLayoutInRect:]_block_invoke + 136
12 UIKitCore                      0x43ce3c -[UICollectionViewData validateLayoutInRect:] + 1252
13 UIKitCore                      0x2878c -[UICollectionView layoutSubviews] + 220
14 UIKitCore                      0x4420 -[UIView(CALayerDelegate) layoutSublayersOfLayer:] + 1992
15 QuartzCore                     0x9f30 CA::Layer::layout_if_needed(CA::Transaction*) + 500
16 QuartzCore                     0x1d4ac CA::Layer::layout_and_display_if_needed(CA::Transaction*) + 148
17 QuartzCore                     0x2e8d8 CA::Context::commit_transaction(CA::Transaction*, double, double*) + 444
18 QuartzCore                     0x5de80 CA::Transaction::commit() + 648
19 UIKitCore                      0x4fd4f0 _UIApplicationFlushCATransaction + 84
20 UIKitCore                      0x64dd94 _UIUpdateSequenceRun + 84
21 UIKitCore                      0xcb2894 schedulerStepScheduledMainSection + 144
22 UIKitCore                      0xcb1df0 runloopSourceCallback + 92
23 CoreFoundation                 0xd3128 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 28
24 CoreFoundation                 0xdf7b4 __CFRunLoopDoSource0 + 176
25 CoreFoundation                 0x645e8 __CFRunLoopDoSources0 + 244
26 CoreFoundation                 0x7a0d4 __CFRunLoopRun + 828
27 CoreFoundation                 0x7f3ec CFRunLoopRunSpecific + 612
28 GraphicsServices               0x135c GSEventRunModal + 164
29 UIKitCore                      0x39cf58 -[UIApplication _run] + 888
30 UIKitCore                      0x39cbbc UIApplicationMain + 340
31 HerdBoss                       0xced0 main + 16 (main.m:16)
32 ???                            0x1dd1b0dec (Missing)

The other threads, for reference:

Fatal error: 👮 Main thread was blocked for 2.00s 👮
Crashed: WatchDog
0  libswiftCore.dylib             0x373e4 _assertionFailure(_:_:file:line:flags:) + 264
1  Watchdog                       0x8328 $s8WatchdogAAC9threshold10strictModeABSd_SbtcfcyycfU_TA + 204
2  Watchdog                       0x8348 $sIeg_IeyB_TR + 28
3  Watchdog                       0x8c1c $s8Watchdog10PingThread33_F0B684E1C455B9DE2999D4562E567E94LLC4mainyyF + 816
4  Watchdog                       0x8c3c $s8Watchdog10PingThread33_F0B684E1C455B9DE2999D4562E567E94LLC4mainyyFTo + 28
5  Foundation                     0x5b524 __NSThread__start__ + 716
6  libsystem_pthread.dylib        0x16b8 _pthread_start + 148
7  libsystem_pthread.dylib        0xb88 thread_start + 8


com.apple.uikit.eventfetch-thread
0  libsystem_kernel.dylib         0xca4 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x13b74 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x13e4c mach_msg_overwrite + 540
3  libsystem_kernel.dylib         0x11e8 mach_msg + 24
4  CoreFoundation                 0x79024 __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x7a250 __CFRunLoopRun + 1208
6  CoreFoundation                 0x7f3ec CFRunLoopRunSpecific + 612
7  Foundation                     0x41fb4 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212
8  Foundation                     0x41e9c -[NSRunLoop(NSRunLoop) runUntilDate:] + 64
9  UIKitCore                      0x4cfcc8 -[UIEventFetcher threadMain] + 416
10 Foundation                     0x5b524 __NSThread__start__ + 716
11 libsystem_pthread.dylib        0x16b8 _pthread_start + 148
12 libsystem_pthread.dylib        0xb88 thread_start + 8


com.google.firebase.crashlytics.MachExceptionServer
0  HerdBoss                       0x1f5b8c FIRCLSProcessRecordAllThreads + 393 (FIRCLSProcess.c:393)
1  HerdBoss                       0x1f5f6c FIRCLSProcessRecordAllThreads + 424 (FIRCLSProcess.c:424)
2  HerdBoss                       0x1ed440 FIRCLSHandler + 34 (FIRCLSHandler.m:34)
3  HerdBoss                       0x1efc18 FIRCLSMachExceptionServer + 521 (FIRCLSMachException.c:521)
4  libsystem_pthread.dylib        0x16b8 _pthread_start + 148
5  libsystem_pthread.dylib        0xb88 thread_start + 8


com.apple.NSURLConnectionLoader
0  libsystem_kernel.dylib         0xca4 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x13b74 mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x13e4c mach_msg_overwrite + 540
3  libsystem_kernel.dylib         0x11e8 mach_msg + 24
4  CoreFoundation                 0x79024 __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x7a250 __CFRunLoopRun + 1208
6  CoreFoundation                 0x7f3ec CFRunLoopRunSpecific + 612
7  CFNetwork                      0x2586f0 _CFURLStorageSessionDisableCache + 60900
8  Foundation                     0x5b524 __NSThread__start__ + 716
9  libsystem_pthread.dylib        0x16b8 _pthread_start + 148
10 libsystem_pthread.dylib        0xb88 thread_start + 8


Thread #1
0  libsystem_kernel.dylib         0x11ac __workq_kernreturn + 8
1  libsystem_pthread.dylib        0xe28 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0xb7c start_wqthread + 8


Thread #2
0  libsystem_kernel.dylib         0x11ac __workq_kernreturn + 8
1  libsystem_pthread.dylib        0xe28 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0xb7c start_wqthread + 8


Thread #3
0  libsystem_kernel.dylib         0x11ac __workq_kernreturn + 8
1  libsystem_pthread.dylib        0xe28 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0xb7c start_wqthread + 8

Thread #4
0  libsystem_kernel.dylib         0x11ac __workq_kernreturn + 8
1  libsystem_pthread.dylib        0xe28 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0xb7c start_wqthread + 8

Thread #5
0  libsystem_kernel.dylib         0x11ac __workq_kernreturn + 8
1  libsystem_pthread.dylib        0xe28 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0xb7c start_wqthread + 8

Thread #6
0  libsystem_kernel.dylib         0x11ac __workq_kernreturn + 8
1  libsystem_pthread.dylib        0xe28 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0xb7c start_wqthread + 8
Accepted Answer

This might be a bug in iOS 16/17 (all the users reporting the problem were on iOS 16.x).

When I was running iOS 16 on my sim and using the iOS 16-compatible version of Xcode, the freeze was never happening to me. iOS 17 got released along with a new version of Xcode, so I updated my Xcode to the new version (I was using the latest one before this as well). Once I did that, I started getting the UI to freeze up for me in the simulator! Woo!

Now that it was happening to me, I was able to run Instruments and track the stacktrace where the UI was locking up more precisely. I got a slightly different stacktrace which referenced a UICalendarView before the UICollectionView stack frames. (I forgot to capture a copy to paste here.) I wasn't familiar with UICalendarView but it appears to be a new replacement for UIDatePicker.

I finally found that viewWillLayoutSubviews was being called over and over, forever. I wasn't changing the size of anything in my UI during those calls... but deep in the iOS code, it was. Something in the UICalendarView code was making some kind of layout change every time layoutSubviews was called, so the system would call viewWillLayoutSubviews again and again, forever because something kept changing.

My UIDatePicker would start off-screen in my storyboard layout and then when it was needed, I would change the constraints in my layout to be smaller and that would cause the UIDatePicker to slide onto the screen.

The only workaround that I found to avoid this weird iOS bug, was to remove the UIDatePicker from my storyboard and then create one in code instead. And instead of doing it during initial creation of the view, I deferred the creation until just before I was going to show the UIDatePicker. And then whatever tickled the weird bug in UICalendarView just didn't happen.

How to find freeze that doesn't happen to me?
 
 
Q