EAAccessoryManager crashes

Hi,

The company produces MFI-certified devices. A lot of users are affected. it happens when users are in the background.

The crash happens in NSArray; it looks like the array is out of bounds to which EAAccessoryManager tries to access.

The only assumption I have is that something is wrong with Threads. Maybe we are blocking a Thread somewhere, or EAAccessory should always work on the Main Thread but we switch streams to another thread.

It is hard to believe that Apple could have such a simple bug. If it were Apple's bug, other companies who produce MFI devices could also experience the identical bug, but I don't see anyone raising the identical issue.

The issues don't have a relation to iOS versions; it happens for us from iOS 13 till iOS 18.

Thanks in advance, Eugene.

Incident Identifier: 9071D48D-2B8C-4636-90E0-8D4BCD382C6A
Distributor ID:      com.apple.AppStore
Hardware Model:      iPhone14,2
Process:             Poly Lens iOS [441]
Path:                /private/var/containers/Bundle/Application/AE5D4478-F14C-4CFB-B892-60D90F2F2BBA/Poly Lens iOS.app/Poly Lens iOS
Identifier:          com.plantronics.myheadset
Version:             4.4.3 (15741)
AppStoreTools:       16A242b
AppVariant:          1:iPhone14,2:15
Code Type:           ARM-64 (Native)
Role:                unknown
Parent Process:      launchd [1]
Coalition:           com.plantronics.myheadset [627]

Date/Time:           2024-09-13 09:42:31.4873 +0200
Launch Time:         2024-09-13 08:23:27.2514 +0200
OS Version:          iPhone OS 17.6.1 (21G93)
Release Type:        User
Baseband Version:    3.50.04
Report Version:      104

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Reason: *** -[NSArray initWithArray:range:copyItems:]: range {0, 1} extends beyond bounds for empty array
Termination Reason: SIGNAL 6 Abort trap: 6
Terminating Process: Poly Lens iOS [441]

Triggered by Thread:  10

Last Exception Backtrace:
0   CoreFoundation                	0x1918fcf20 __exceptionPreprocess + 164 (NSException.m:249)
1   libobjc.A.dylib               	0x18979b2b8 objc_exception_throw + 60 (objc-exception.mm:356)
2   CoreFoundation                	0x19188c0ac -[NSArray initWithArray:range:copyItems:] + 1180 (NSArray.m:0)
3   CoreFoundation                	0x1918c39a8 +[NSArray arrayWithArray:] + 60 (NSArray.m:975)
4   ExternalAccessory             	0x1c4beada4 -[EAAccessoryManager _findExtraAccessoriesContainedOnlyInEA:] + 88 (EAAccessoryManager.m:1235)
5   ExternalAccessory             	0x1c4bea630 -[EAAccessoryManager _checkForConnectedAccessories:backgroundTaskIdentifier:] + 144 (EAAccessoryManager.m:1274)
6   libdispatch.dylib             	0x1997a113c _dispatch_call_block_and_release + 32 (init.c:1530)
7   libdispatch.dylib             	0x1997a2dd4 _dispatch_client_callout + 20 (object.m:576)
8   libdispatch.dylib             	0x1997a5f6c _dispatch_queue_override_invoke + 928 (queue.c:4967)
9   libdispatch.dylib             	0x1997b4894 _dispatch_root_queue_drain + 392 (queue.c:7136)
10  libdispatch.dylib             	0x1997b509c _dispatch_worker_thread2 + 156 (queue.c:7204)
11  libsystem_pthread.dylib       	0x1ee34c8f8 _pthread_wqthread + 228 (pthread.c:2709)
12  libsystem_pthread.dylib       	0x1ee3490cc start_wqthread + 8 (:-1)

I can't attach full log cause Forum showed this error: This post contains sensitive language. Please revise it in order to continue.

Please try again. See Posting a Crash Report for specific advice on how to do this and a workaround if you continue to have problems.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks!

Full log is here:

It is hard to believe that Apple could have such a simple bug. If it were Apple's bug, other companies who produce MFI devices could also experience the identical bug, but I don't see anyone raising the identical issue.

If you haven't already, please file a bug on this then post the bug back here. The bug is not simple, but we might be able to do something to handle this better.

Looking at the crash itself, let me start with this error:

Exception Reason: *** -[NSArray initWithArray:range:copyItems:]: range {0, 1} extends beyond bounds for empty array

What this error actually means is that you passed in a range that's outside of the contents of the target array. In this case, you'd be passing in the range {0, 1} to an empty array. The problem here is that this is NOT what EAAccessory did. What they called was:

3   CoreFoundation                	0x1918c39a8 +[NSArray arrayWithArray:] (in CoreFoundation) + 60

And what arrayWithArray actually did was:

return [[[self alloc] initWithArray:array range:NSMakeRange(0, [array count]) copyItems:NO] autorelease];

On it's own, that code should never generate this failure since, by defintion, the range of any array is {0, count}. Also, just to cover the edge case, the issue isn't that the array is NULL, as initWithArray will actually work fine with a NULL array and a null array would have generated {0, 0}.

That leads to here...

The only assumption I have is that something is wrong with Threads.

Yes, threads are definitely. The only way I can see this happening is that something deleted the object from that array at EXACTLY the moment the array was being created. However, that does imply some very specific timing and I'm no sure what would have created that.

Moving to here:

Maybe we are blocking a Thread somewhere, or EAAccessory should always work on the Main Thread but we switch streams to another thread.

Yes, though it's not the main thread itself that's at issue. A few things I'd be concerned about here:

  • EAAccessory calls it's delegate on the main thread, so using it on another thread means you're creating race conditions between those two thread.

  • NSStream is a VERY old API which, depending on how it's used, is heavily reliant on a having a working run loop. That basically means that using it on a GCD thread could cause weird problems, depending on exactly how it's used.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

@DTS Engineer

First of all thanks for your answer.

In the message before yours I have posted full log.

The interesting stuff for me, that NSArray is handling by EAAccessory, it is part of EAAccessory framework, not my Array. So I try to understand what I can do with thread to affect Main Thread in that way that Object could be deleted in that moment when it was created.

And usually it is happens in background. And btw, it is only visible on PROD, so when optimization is turned on. Our QA has never observed the bug.

Thanks, Eugene.

In the message before yours I have posted full log.

Yep, that's what I based my earlier post on. Unfortunately, all I can really talk about here is what happened in "our" code. There's clearly plenty of other thread activity happening as well which is probably involved in creating the problem, but I can't really be any more specific than that.

The interesting stuff for me, that NSArray is handling by EAAccessory, it is part of EAAccessory framework, not my Array. So I try to understand what I can do with thread to affect Main Thread in that way that Object could be deleted in that moment when it was created.

Yes, but that array is holding the systems references to underlying object that EAAccessory manipulates. I'm not sure of what's happening that actually creates the crash, but that's my best guess from comparing the crash log and our code.

Also, to clarify, I do think our code has some "involvement" here, as that array shouldn't really be able to be put into that state. However, that doesn't mean you don't also have an issue here. Our fix for this kind of threading issue often ends up exposing other issue that were previously hidden by the current crash, assuming it doesn't simply result in a different crash*.

*As a more direct and dramatic example of this, my recommendation to any developer who's app manages to cause a kernel panic is:

  1. File a bug immediately. The kernel should never panic, so all kernel panics are bugs.

  2. Figure out what you're doing that's causing the panic and stop immediately.

The reason for #2 is quite simple. I said the kernel shouldn't panic, I didn't say it had to keep running your app. As far as the kernel is concerned terminating your process is a perfectly reasonable way to avoid panic'ing, but that "fix" won't really improve your apps user experience.

And usually it is happens in background. And btw, it is only visible on PROD, so when optimization is turned on.

FYI, you can change the compiler optimization level for debug builds. It defaults to "0" because optimizations like instruction reordering confuse the debugger*, but you can still follow the "flow" pretty well even at "5".

Also, if you haven't already, take a look at "Diagnosing memory, thread, and crash issues early" and do some testing with those tools, particularly the Thread Sanitizer. I can't promise they'll find anything (unfortunately, bug like this are hard...), but these issues are hard enough to investigate that a tool that finds even SOME problems can be a huge help.

Our QA has never observed the bug.

Yes, that's pretty common. The nature of threading failures is that they often require very specific circumstances to reproduce and, in most cases, you won't known what those are until after you've found the problem. However, two things you can look at here:

  1. If you're in contact with any users experiencing this issue or have any other data about them, what makes them "different" than other user? Why makes them "special" compared to everyone else? One thing I particularly like to look at here is the crash timestamps, looking for things like how long the app was running, what time the crash occurred, and what timezone the crash was in (which gives an indication of "where" the users might be).

  2. What is your QA process actually testing. For accessory apps in particular, I think there are two different "extremes" that testing should be focused on:

  • Stressing the accessory and app to artificially induce failure. For example, leaks or other issues are more likely at "transition" points, so a testing process that forces the app to connect/disconnect hundreds/thousands of times can be a lot more useful than simply testing normal app use.

  • Replicating real world usage in a way that gets you good data about what actually went wrong. The specifics of how this work depend on what you're accessory actually is but one crash from an app build that generates detailed activity logs or from a developers who's aware of your app internal implementation details can be FAR more useful than a standalone crash log.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

EAAccessoryManager crashes
 
 
Q