What causes EXC_BAD_ACCESS KERN_INVALID_ADDRESS with tagged pointers?

I have a growing collection of iOS crash reports (3 to 4 per day) which crash inside calloc or malloc, saying "EXC_BAD_ACCESS KERN_INVALID_ADDRESS" and then listing a pointer address which either looks like a tagged pointer, like

0x0000000000000026, 0x000000000000001c, 0x000000000000005a

or pointers that look suspiciously low, like

0x0000000000000010, 0x0000000000000020, 0x0000000000000030.


Stats say it's happening for 5% of users, we've only been able to trigger it in the office once, and the debugger wasn't attached. So I haven't been able to trap it with zombies, or address sanitizer.


So my questions are:

1) Are the 0x10, 0x20, 0x30 also tagged pointers? or just really low value real pointers, and thus likely a different problem?

2) What could cause tagged pointers to actually get free'd so that they'd ever end up available for malloc or calloc? We do use both Obj-c and swift, but I thought the compiler would handle conversions between obj-c tagged objects like NSNumber's and NSDate's and swift types. Could a lack of @objc on an extension's override of a protocol method cause the conversion to not happen? Should I be looking into code that accidently calls CFRelease on an NSNumber or NSDate? I don't have any -f-no-arc files in the project, but I do have a few pre-compiled 3rd party libraries in the app.

2b) Which other NSObject subclasses can be tagged pointers?

3) How do I discover where the instance was allocated, or where it was free'd without either releasing the modified app to production and without having it hooked up to the debugger? I can have our qa person tap the app incessantly, but it would work better if we had a way to capture the addresss sanitizer or zombie output and send it up via fabric crash reports? Can I catch the contents that zombies or address sanitizer would normally print to the console and include it in the crash report? I have a synchronous way to add arbitrary lines of text to the crash report if I can intercept it before crashlytics gets it, but my experience with that is limited to intercepting NSExceptions, and this is way deeper than NSException.

3b) Is there maybe a way to get it to crash when the bad value is free'd instead or reused so our internal qa builds can at least narrow down the scope of what got released wrong, instead of "literally maybe anything"?

Replies

Why do you post the exact same message, under 2 different identities ?


Please, delete one of them.

You don't have any evidence that this is anything to do with tagged pointers. What you have evidence of is corrupted pointers, which suggests a memory corruption issue. (Even if the ultimate source of those particular hex values was a tagged pointer, it would take some kind of memory corruption to get them to be used in a way that would crash like this.)


It might be useful if you could show some more levels of backtrace from the thread that crashed. That might shed some light on what's gone wrong.


The good news here is that 5% is a pretty high failure rate for issues like this. It's (relatively) easy to reproduce the problem at that rate.

I posted a quesiton, and 4 days later Apple wouldn't show it to anyone, saying it still needed to be moderated. They didn't say why. I considered that it might be because it was a coorporate id, and had forwarded a moderation request to some supervisor who had better things to do. So I posted under a personal account. That one sat for days unposted also requiring moderation. It turns out it just takes the better part of a week to approve asked questions. I'll go delete one, especially since, in the mean time, a poster on SO pointed out that the pointer values weren't pointer values, but register addresses.

Yeah, it would be great if we could make it happen in the lab, but so far, no success, and I've tried several dozen times. I'm now investigating the posibility that these only happen when the app is in the background.

I get 4 crash reports per day, always in slightly different stack traces, and always in nothing but system code. Lots of them happen in networking or UI code (that's about all the app does), most often under calloc or malloc, and also somewhat commonly under URLResponse::~URLResponse()

>> saying it still needed to be moderated


The forum software is a bit quirky, and sometimes decides automatically to moderate posts that have nothing wrong with them. AFAICT, it'll stay in that state until someone with moderator privileges happens to notice. In that situation, your best strategy is to make a short additional post, saying that your other post was moderated. That will increase your visibility, and maybe get your post out of moderation.


(FWIW, the forum software used to moderate any post that used the word 'goal'. No one seemed to understand why. Is it a rude word in some language?)


>> register addresses


Wat???


They're just numbers.


>> make it happen in the lab, but so far, no success


Yeah, that's unfortunate. You might have to add code to write a log file that users can send back to you, with some kind of trace or state information that would help identify what conditions are needed to trigger the crash.

It turned out to be an error in the Xcode 9.? Swift compiler. I guess everyone knows that now. Merely upgrading to Xcode 10 fixed the problem.