Xcode 11 dictionary doesn't like concurrent access on iOS

FB6684220


my app that worked fine under Xcode 10 crashes under Xcode 11 (beta 3). I distilled the crash down to this: when accessing dictionary concurrently some of the acceses crash. interestingly it happens only under iOS, not under macOS. FB6684220


========


import Foundation


var dict: [String : String] = [:]


func test() {

Thread.detachNewThread {

var counter = 0

while true {

counter += 1

let key = String.random

dict[key] = .random

usleep((0...1000).randomElement()!)

}

}


Thread.detachNewThread {

var counter = 0

while true {

counter += 1

let key = String.random

let value = dict[key]

usleep((0...1000).randomElement()!)

}

}

}


extension String {

static var random: String {

let s = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. "

let offset = (0 ... 100).randomElement()!

let len = (1 ... 100).randomElement()!

return String(s[s.index(s.startIndex, offsetBy: offset) ... s.index(s.startIndex, offsetBy: offset + len)])

}

}

Accepted Reply

It is not safe to share unprotected mutable data (like

dict
) between threads. The fact that this didn’t crash in Xcode 10 is just an artefact of its implementation.

Fortunately there’s a way to flush out latent threading bugs like this one, namely, the thread sanitiser. If you run your code under the thread sanitiser, it immediately fails.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Replies

this article is interesting but 8 years old: "Real-time audio programming 101: time waits for nothing" by Ross Bencina.

if it's no longer applicable to apple platforms - please shout.

Any developers who based their code on Apple's sample code would have given up years ago.

If you run any sufficiently large multi-threaded program under the thread sanitiser, it will flag problems. Some of those problems with be false positives, but the vast majority of them are folks not following the rules. If the program actually works, there’s two possible results:

  • The program will work all the time on its current target platform because that platform has a sufficiently strong memory model (for example, Intel).

  • The program can fail at runtime on its current target platform, but such failures are rare.

However, this doesn’t make the programs correct, and that incorrectness may results in problems in the future. For example:

  • The tools may evolve to exploit undefined behaviour guarantees.

  • The program may be ported to a different platform, with a weaker memory model.

  • The target platform may evolve to exploit it’s existing weak memory model. For example, the Arm CPU on the original iPhone was pretty much sequential, but the Arm CPU on current iPhones supports a high degree of reordering.

None of this stuff is Apple specific and I strongly encourage you to research it via non-Apple channels. There’s a bunch of good info out there, including Memory Consistency Models: A Tutorial.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

thanks Quinn, that makes sense.


i haven't seen this previously in a sample code, so i'm going to ask DTS (via a tech support incident) how to address your bullet point #2 right (coordinating buffer contents with it's position) from inside an audio callback. let's see what they answer.


one followup question. i can see that i can easily fool thread sanitizer by doing this:


int one = 1;

memmove(&flag, &one, sizeof(one));


instead of:

flag = 1;


i assume this a current limitation (or a bug) of thread sanitizer and memmove doesn't actually do anything special (like flushing caches or putting memory fences) than the simple assignment does, correct?


please also comment on the messages which might be lost above (whether the priority inversion problem is fixed on apple platform and if so does or doesn't it have any impact / mitigation on what we now can call from RT threads; and whether the info in the old article of Ross Bencina is still applicable to apple platforms as it (mostly) basing its rationale on the priority inversion issue).


my current understaning is that the prioroity inversion problem is the main spanner in the works, and if it is solved then not only this opens a possibility of using locks inside RT threads, but do other things that were previously impossible (e.g. using malloc or swift/obj-c runtime inside RT threads). please share your thoughts on this.

coordinating buffer contents with it's position

My assumption is that you’ll need a memory barrier but, honestly, I’m not sufficiently confident of my understanding of memory barriers to offer concrete advice here (I’ve seen lots of folks give lots of bad advice about memory barriers, which makes me reluctant to want to contribute to that corpus).

i assume this a current limitation (or a bug) of thread sanitizer and memmove doesn't actually do anything special (like flushing caches or putting memory fences) than the simple assignment does, correct?

Correct.

With regards priority inversion, most of the built-in locking primitives provide a priority boost to the thread holding the lock when there’s a high-priority thread waiting for it. You can learn more about this in the Lock Ownership section of WWDC 2017 Session 706 Modernizing Grand Central Dispatch Usage.

Whether that’s sufficient for you to use locks in the context of an audio rendering thread, I don’t know. Even if it were, I’d only use that for lightweight stuff, for example, your ring buffer. Allocating memory is a potentially heavyweight operation and I’d be very leery about doing that in an audio rendering thread.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

> My assumption is that you’ll need a memory barrier …


yes, looks like it. though i never saw it used for audio on ios/mac..


>> i assume this a current limitation (or a bug) of thread sanitizer and memmove doesn't actually do anything special (like flushing caches or putting memory fences) than the simple assignment does, correct?

> Correct.


i remember BlockMove was doing something special about caches :-)


> With regards priority inversion, most of the built-in locking primitives provide a priority boost to the thread holding the lock when there’s a high-priority thread waiting for it. You can learn more about this in the Lock Ownership section of WWDC 2017 Session 706 Modernizing Grand Central Dispatch Usage.


thanks, will definitely watch.


i created a simple test application that tests bullet point #2 statement (the necessity of coordinating accesses to buffer contents and its position). it is a 370 line all-in-one-source-file app -- no nib files -- for iOS and macOS. on iOS the app has some minimal UI, on macOS it is a console app that prints some info every second. the code is mainly “C” as far as audio is concerned, notable exception is std::atomic<int> used for ring buffer positions to avoid their data races. no attempt is done in code to coordinate buffer content and its position. the app works like this:


- the app gradually generates and fills the ring buffer with audio data (a chord of three notes). this happens in the main thread. the assumption is that in order for the issue to happen the ring buffer size shall be small enough and so shall be the filling chunks. i tested it with filling duration of down to 10ms at 48kHz (so 480 samples == 2K bytes). also with big filling chunks / ring buffer (e.g. 1/3 sec / 1 sec).


- in the IO callback (real time audio thread) the app reads from the ring buffer and plays it.


- to ensure the data is not damaged (the actual test of the bullet point #2 statement) the app also generates the relevant chunk of audio data “as it should be” on the fly and compares it with what came from the ring buffer - it asserts the two are equal, so in debug build the app will break.


- the main UI on iOS / or console output on mac shows various information (sample rate, etc) along with the “error count”. should this error count be anything but zero, ever - there is indeed a need to coordinate accesses of ring buffer contents and its position.


- if you want to compile this app i can either provide the project or just create a new project for iOS app and remove everything including the “Main storyboard file base name” reference in it’s plist. as for macOS target - use console app. i found i had to manually add frameworks (in the Xcode settings pane) when C++ is used in the app. also enable background audio in iOS target if you want the app to work in background.


- note that error checking in the app is minimal - the app asserts on errors.


i was not able to surface the problem #2 so far, tested debug and release builds on iPhone X, iPhone 6s, iPad Pro / MacBook Pro. will soon test the app on the most recent iPhone and update here if it is any different.


i foresee this awkward conversation should i talk about this app with DTS:


>>>>>>>>>>>>>>>

ME: please see the app, it works but it shall not.


DTS: we tested your app and we weren’t able reproduce any issues. do you know on what hardware and/or under what circumstances it shall misbehave?


ME: no, but it may fail when iPhones are ported to alpha CPU or when ARM moves to a weaker memory model or when tools are evolved or maybe on the current hardware in other cases which i haven’t found - please help me finding those. Quinn told me that to play by the rules i shall synchronise the access of the buffer and its position otherwise i am (and always was) in trouble without realising it.


DTS: you shall talk to Quinn then. from what we see the app works ok and we are unable to find a case when it doesn’t work so no changes are needed. following the YAGNI principle, if the app breaks in the future - you fix it right there and then in the future. we also compared what you do in the app to other relevant sample code and found no significant differences. having said that we have to close this DTS incident.

<<<<<<<<<<<<<<


let’s see what they actually say.


==== the app ====

475 lines skipped, probably too big for this forum. will send it to those who want it.


the most interesting fragment looks like this:


static OSStatus renderCallback(void* refCon, AudioUnitRenderActionFlags* actionFlags, const AudioTimeStamp* ts, UInt32 element, UInt32 numFrames, AudioBufferList* ioData) {

..

ringRead(data, size);

generateAudio(&phase, data2, size);

assert(memcmp(data, data2, size) == 0);

..

}