Variable not captured unless referencing explicitly from local assignment

I'm trying to modify a shared __block BOOL variable for the purposes of flagging of execution to proceed. I implemented it successfully by creating an object that encapsulates the BOOL variable, but when sending the block inline, there are issues--I must create a local variable for the block and explicitly reference the BOOL variable inside of it (marked 1A in code sample). Any ideas how to make the - (void)primitive_cancellable_dispatch_after:(dispatch_time_t)delay block:(MyBlock)block stop:(BOOL*)stop method work without this requirement from the call site? It seems that the variable is not being captured unless it is being assigned locally and referenced. In general, at what point is the variable captured if not being explicitly referenced locally?

    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC));

    __block BOOL stop;
	
	// =============== 1A ===============
    MyBlock primitiveBlock = ^{
        NSLog(@"Ran primitive implementation via assignment. (stop: %@ / %p)", @(stop), &stop); // stop only "captured" when referencing within a locally assigned block, if this line is commented out, stop isn't updated
    };
	// =============== 1A ===============
	
    
    [self primitive_cancellable_dispatch_after:delay block:^{
        NSLog(@"Ran primitive implementation via inline. (stop: %@ / %p)", @(stop), &stop);
    } stop:&stop];
    
    __block Task *task = [self reference_cancellable_dispatch_after:delay block:^{
        NSLog(@"Ran reference implementation: (task.shouldCancel: %@)",@(task.shouldCancel));
    }];
        
    task.shouldCancel = NO; // works as expected
    stop = NO; // stop isn't updated as expected, when checking value within "primitive_cancellable_dispatch_after", always returns no
}

- (void)primitive_cancellable_dispatch_after:(dispatch_time_t)delay block:(MyBlock)block stop:(BOOL*)stop {
    dispatch_after(delay, dispatch_get_main_queue(), ^{
        BOOL shouldStop = *stop; // stop doesn't seem to resolving correctly, stop is always "NO", unless assigning a local block outside
        NSLog(@"Checking primitive stop (shouldStop: %@ / %p)", @(shouldStop), stop);
        if(!shouldStop && block) {
            block();
            NSLog(@"Checking primitive stop (shouldStop: %@ / %p)", @(shouldStop), stop);
        }
    });
}

/*@interface Task : NSObject
@property (nonatomic, assign) BOOL shouldCancel;
@property (nonatomic, strong) Task *task;
@end

@implementation Task
@end*/
- (Task*)reference_cancellable_dispatch_after:(dispatch_time_t)delay block:(MyBlock)block {
    Task *task = [Task new];
    
    dispatch_after(delay, dispatch_get_main_queue(), ^{
        if(!task.shouldCancel && block) {
            block();
        }
    });
    
    return task;
}

It’s hard to tell what’s going on here because:

  • You don’t show any code that sets stop to true.

  • There’s a bunch of infrastructure that’s specific to your app (like Task) that you don’t discuss.

  • Is reference_cancellable_dispatch_after involved in this at all? Neither it nor the block you pass to it uses stop, so I’m confused why you didn’t edit it out.

Any chance you can cut this down a minimal test that doesn’t rely on any of your infrastructure?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Task was just a wrapper for a BOOL showing implementation that worked, but you make a good point that it was somewhat irrelevant to what I was trying to get to. Here it is distilled down:

typedef void(^MyBlock)(void);

- (void)primitiveStopBlockTest {
    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC));

    __block BOOL stop;
    [self primitive_cancellable_dispatch_after:delay block:^{
        NSLog(@"Ran primitive implementation with calling inline. (stop: %@ / %p)", @(stop), &stop);
    } stop:&stop];
    
    stop = YES;
}

- (void)primitiveStopBlockWithLocalAssignmentTest {
    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC));

    __block BOOL stop;
    MyBlock blockWithLocalAssignment = ^{
        NSLog(@"Ran primitive implementation with local assignment. (stop: %@ / %p)", @(stop), &stop);
    };
    [self primitive_cancellable_dispatch_after:delay block:blockWithLocalAssignment stop:&stop];
    
    stop = YES;
}

- (void)primitive_cancellable_dispatch_after:(dispatch_time_t)delay block:(MyBlock)block stop:(BOOL*)stop {
    dispatch_after(delay, dispatch_get_main_queue(), ^{
        BOOL shouldStop = *stop;
        
        if(block && !shouldStop) {
            NSLog(@"Executing block. (shouldStop: %@ / %p) (block: %p)", @(shouldStop), &shouldStop, &block);
            block();
        }
        else {
            NSLog(@"Block execution cancelled. (shouldStop: %@ / %p) (block: %p)", @(shouldStop), &shouldStop, &block);
        }
    });
}

Output for primitiveStopBlockTest is:

Executing block. (shouldStop: 0 / 0x16bc04bef) (block: 0x600003095340)
Ran primitive implementation with calling inline. (stop: 1 / 0x600003edaf18)

Output for primitiveStopBlockWithLocalAssignmentTest is:

Block execution cancelled. (shouldStop: 1 / 0x16b574bef) (block: 0x600002adcd70)

I am just curious as to why stop is getting set successfully in the primitiveStopBlockWithLocalAssignmentTest function but not the primitiveStopBlockTest function.

This is working on my machine. Specifically, I did the following:

  1. In Xcode 12.5 on macOS 11.4 I created a new project from the macOS > Command Line Tool template.

  2. I added a dummy class Main and put your code inside it.

  3. I replaced the NSLog in main with this:

    NSLog(@"will start");
    Main * m = [[Main alloc] init];
    [m primitiveStopBlockTest];
    [m primitiveStopBlockWithLocalAssignmentTest];
    dispatch_main();
    
  4. I chose Product > Run.

This is what I see printed:

2021-06-18 10:50:07.912180+0100 xxot12[5505:165229] will start
2021-06-18 10:50:09.913694+0100 xxot12[5505:165798] Block execution cancelled. (shouldStop: 1 / 0x70000d478bbf) (block: 0x109a04750)
2021-06-18 10:50:09.913953+0100 xxot12[5505:165798] Block execution cancelled. (shouldStop: 1 / 0x70000d478bbf) (block: 0x109a04cd0)

That’s correct, right?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I setup a new project in the same method you described and ran with different results. I am on Xcode 12.5, macOS 11.4 as well (using MacBook Air (M1, 2020)).

main.m:

#import <Foundation/Foundation.h>

typedef void(^MyBlock)(void);
@interface Main : NSObject
- (void)primitiveStopBlockTest;
- (void)primitiveStopBlockWithLocalAssignmentTest;
@end

@implementation Main
- (void)primitiveStopBlockTest {
    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC));

    __block BOOL stop;
    [self primitive_cancellable_dispatch_after:delay block:^{
        NSLog(@"Ran primitive implementation with calling inline. (stop: %@ / %p)", @(stop), &stop);
    } stop:&stop];
    
    stop = YES;
}

- (void)primitiveStopBlockWithLocalAssignmentTest {
    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC));

    __block BOOL stop;
    MyBlock blockWithLocalAssignment = ^{
        NSLog(@"Ran primitive implementation with local assignment. (stop: %@ / %p)", @(stop), &stop);
    };
    [self primitive_cancellable_dispatch_after:delay block:blockWithLocalAssignment stop:&stop];
    
    stop = YES;
}

- (void)primitive_cancellable_dispatch_after:(dispatch_time_t)delay block:(MyBlock)block stop:(BOOL*)stop {
    dispatch_after(delay, dispatch_get_main_queue(), ^{
        BOOL shouldStop = *stop;
        
        if(block && !shouldStop) {
            NSLog(@"Executing block. (shouldStop: %@ / %p) (block: %p)", @(shouldStop), &shouldStop, &block);
            block();
        }
        else {
            NSLog(@"Block execution cancelled. (shouldStop: %@ / %p) (block: %p)", @(shouldStop), &shouldStop, &block);
        }
    });
}
@end

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        NSLog(@"will start");
        
        Main *m = [[Main alloc] init];
        [m primitiveStopBlockTest];
        [m primitiveStopBlockWithLocalAssignmentTest];
        dispatch_main();

    }
    return 0;
}

Output:

2021-06-18 17:31:36.054948-0700 Test[12721:383872] will start
2021-06-18 17:31:38.234779-0700 Test[12721:384213] Executing block. (shouldStop: 0 / 0x16fe86b9f) (block: 0x10067a580)
2021-06-18 17:31:38.235058-0700 Test[12721:384213] Ran primitive implementation with calling inline. (stop: 1 / 0x10067a5a8)
2021-06-18 17:31:38.235157-0700 Test[12721:384213] Block execution cancelled. (shouldStop: 1 / 0x16fe86b9f) (block: 0x10067acb0)

I will email you the project itself as well.

Your project works the same as mine on my Intel Mac but fails as you’ve reported on my Apple silicon Mac. I cut this down even further (see the code below) and it’s revealing some strange behaviour, behaviour that varies between platforms and compiler settings. I haven’t dug into all the details and I’m reluctant to do so because your code has a clear undefined behaviour problem.

Consider this output from my Apple silicon Mac:

2021-06-21 11:54:08.023946+0100 Test[13327:686828] > -dispatchAfterTest, stop: 0, &stop: 0x7ffeefbff2c0
2021-06-21 11:54:08.025797+0100 Test[13327:686828] > -dispatchAfter:block:stop:, stop: 0, &stop: 0x7ffeefbff2c0
2021-06-21 11:54:08.026488+0100 Test[13327:686828] < -dispatchAfterTest, stop: 1, &stop: 0x7b080002a3f8
2021-06-21 11:54:10.023157+0100 Test[13327:687477] ! -dispatchAfter:block:stop:, stop: -48, &stop: 0x7ffeefbff2c0

There are two threads involved here:

  • 686828 is the main thread

  • 687477 is a Dispatch worker thread

Now consider the last two log points. There are on different threads and the only reason they can’t collide in this test project is the 2 second delay. That delay eliminates the problem in practice but it doesn’t eliminate it in theory. The C language spec (which is all that you have to go on when dealing with Objective-C) is clear that multiple threads accessing the same value without locking [1] results in undefined behaviour.

This is particularly important when you work on a weak memory architecture CPU, like M1. Without explicit locking [1] you really will see weird problems on M1.


Now, there’s also some other weird things going on here:

  • The address of stop returns two different values, one of which is on the stack!

  • The thread sanitiser doesn’t pick up this problem.

I think these are related. My guess is that ARC is not copying the block at the right time, so the code on the main thread is seeing the on-stack version of stop and the code on the secondary thread is seeing the version after it’s been moved to the heap. However, I didn’t dig into this any further because of the above-mentioned undefined behaviour issue.


So, my specific advice here is that you change stop from being a simple BOOL to something that’s appropriate to be shared between threads (something protected by locks, an atomic, a Dispatch ‘or’ source, or whatever). I suspect that you’ll end up fixing this problem in the process.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Or appropriate use of atomics.

#import <Foundation/Foundation.h>

typedef void(^MyBlock)(void);

@interface Main : NSObject
@end

@implementation Main

- (void)dispatchAfterTest {
    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC));

    __block BOOL stop = NO;
    NSLog(@"> -dispatchAfterTest, stop: %d, &stop: %p", (int) stop, &stop);
    [self dispatchAfter:delay block:^{
        NSLog(@"! -dispatchAfterTest, stop: %d, &stop: %p", (int) stop, &stop);
    } stop:&stop];
    
    stop = YES;
    NSLog(@"< -dispatchAfterTest, stop: %d, &stop: %p", (int) stop, &stop);
}

- (void)dispatchAfter:(dispatch_time_t)delay block:(MyBlock)block stop:(BOOL*)stopPtr {
    NSLog(@"> -dispatchAfter:block:stop:, stop: %d, &stop: %p", (int) *stopPtr, stopPtr);
    dispatch_after(delay, dispatch_get_main_queue(), ^{
        BOOL shouldStop = *stopPtr;
        NSLog(@"! -dispatchAfter:block:stop:, stop: %d, &stop: %p", (int) *stopPtr, stopPtr);
        if(block && !shouldStop) {
            block();
        }
    });
}

@end

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        Main *m = [[Main alloc] init];
        [m dispatchAfterTest];
        dispatch_main();
    }
    return 0;
}

From my understanding, in theory, our code should be doing everything on the main thread, so why is the undefined behavior of concern? Why is it switching to a different thread in practice when we are sending dispatch_get_main_queue() to the dispatch_queue_t queue parameter of dispatch_after?

In reference to the ARC timing issue—is creating a locally assigned block and referencing the __block variable (as demonstrated in primitiveStopBlockWithLocalAssignmentTest) forcing the correct timing or is this unrelated? In general, what is the timing and what can we reliably conclude across different architectures?

When you mention "something that’s appropriate to be shared between threads," Apple itself uses primitives in its example of demonstrating block storage. Is there something I'm misunderstanding or should the caveat be added to the documentation?

I had a hunch that it had something to do with some specific quirks I was glossing over, which is what I was alluding to first with the original post with the Task object which was essentially a object wrapper for BOOL. Interestingly though, the property was marked as nonatomic but it still worked fine on my machine.

What originally spawned all this was I that I was originally trying to implement some variation of enumerateObjectsUsingBlock: (documentation) but lack of exact remembering caused me to write the function as sending the stop as part of the function rather than as part of the block. Is it better practice / safer to send primitive variables as part of the block inline vs. as a function parameter?

In regards to undefined behavior, I was always somewhat aware of issues between system architectures in regards to data types, but had not considered it in the context of multithreading so thanks for bringing that to my attention.

Why is it switching to a different thread in practice when we are sending dispatch_get_main_queue to the dispatch_queue_t queue parameter of dispatch_after?

Now that’s a good question, and the answer undermines my conclusion. Lemme explain…

My test project uses dispatch_main. That routine ‘parks’ the main thread, relying on Dispatch queues for subsequent work. However, it turns out that it doesn’t actually park the thread, but rather it terminates the thread. When work shows up on a queue, Dispatch will allocate, or start, a worker thread to perform that work.

And yes, this means that an Apple platforms you can have a process with no threads inside it. Weird.

This behaviour is good because it allows the system to release the resources held by the main thread. However, it can be confusing because the thread used to service the main queue can vary over time. This is why my logging shows different thread IDs.

And those different thread IDs caused me to assume that this work was being done concurrently, and hence my undefined behaviour claim. If, in your real product, you can guarantee that this BOOL is only accessed from the main thread (or main queue) then, you’re right, that’s not undefined behaviour.


In reference to the ARC timing issue—is creating a locally assigned block and referencing the __block variable … forcing the correct timing or is this unrelated?

Probably. To understand what’s going on here you need to understand a little about how Objective-C blocks work. By default these live on the stack, which avoids an allocation in situations, like -enumerateObjectsUsingBlock:, where the block doesn’t ‘escape’ (using a term from Swift here). In MRR, any code that was responsible for escaping a block was also responsible for copying the block to the heap. C code uses _Block_copy for this; Objective-C code can simply send the -copy message to the block.

ARC works hard to automate this. If it sees that a block might escape it does the copy for you. I think you’ve hit a situation where that’s not working properly.


When you mention "something that’s appropriate to be shared between threads," Apple itself uses primitives in its example of demonstrating block storage.

I followed your link and didn’t see a specific problem. The doc has code that uses __block and code that uses queues but it doesn’t seem to mix them in an undefined way.

What specific example is troubling you?


So, anyway, with my threading theory blown out of the water I’m going to fall back to my _Block_copy theory. I think what’s happening here is that ARC is moving the block to the heap when -dispatchAfterTest calls -dispatchAfter:block: but the fact that this call also takes the address of stop is confusing things so that stop ends up pointing to the stack. However, I don’t have enough time to properly test that theory here on DevForums.

If you open a DTS tech support incident I could look into this in more detail but I’m not sure I’d recommend that. I suspect that the final answer here will be that this pattern — having a __block variable that you access by capturing it in an escaping block and via a pointer — is causing problems and that the way to avoid those problems would be to stop doing that.

Which brings me to this…


Is it better practice / safer to send primitive variables as part of the block inline vs. as a function parameter?

I don’t understand this part of your question. Please elaborate.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Got it, I've never actually seen dispatch_main used (as an iOS Developer), from what I am reading, this routine is never really used in any production apps, correct?

ARC works hard to automate this. If it sees that a block might escape it does the copy for you. I think you’ve hit a situation where that’s not working properly.

In regards, to ARC, you mentioned that it works hard to automate things, but it seems this is at the expense of giving up exact control. Is it a bug in the language that it’s not capturing the primitive BOOL variable? In my original example of code, I implemented the Task wrapper object and it seemed the system had no issues seeing changes to the shouldCancel property.

So, my specific advice here is that you change stop from being a simple BOOL to something that’s appropriate to be shared between threads (something protected by locks, an atomic, a Dispatch ‘or’ source, or whatever). I suspect that you’ll end up fixing this problem in the process.

I followed your link and didn’t see a specific problem. The doc has code that uses __block and code that uses queues but it doesn’t seem to mix them in an undefined way.

You mentioned not to use BOOL, and the documentation uses char:

__block char localCharacter;

So, I was just curious as to where you were going with that or if I’m misunderstanding. By primitive, I am referring to things like int, char, BOOL, etc. BOOL is essentially just defined as:

typedef signed char BOOL; 

Objective-C code can simply send the -copy message to the block

Like this? I don't think I've ever seen this done in practice, but it seemed to work.

- (void)primitiveStopBlockWithLocalAssignmentAndCopyTest {
    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC));

    __block BOOL stop;
    [self primitive_cancellable_dispatch_after:delay block:[^{
        NSLog(@"Ran primitive implementation with calling inline. (stop: %@ / %p)", @(stop), &stop);
    } copy] stop:&stop];
    
    stop = YES;
}

I suspect that the final answer here will be that this pattern — having a __block variable that you access by capturing it in an escaping block and via a pointer — is causing problems and that the way to avoid those problems would be to stop doing that.

This is essentially what I have came to the conclusion as well. The paradigm worked for objects but not primitives unless explicitly creating a local block and referencing it, or just manually send the block the copy message. It just seems messy because if I am essentially passing variables through, I had to locally reference the variables and do a log, if I didn't want that, I pretty much had to do a no-op with the variable to quiet the compiler warnings. Overall, I definitely understand workarounds, solutions, etc. in getting what I am trying to do to work, but I am trying to understand fundamentally why the primitive variable was not captured properly to determine if my mental model of how everything works is flawed.

Is it better practice / safer to send primitive variables as part of the block inline vs. as a function parameter? I don’t understand this part of your question. Please elaborate.

Wondering if it's safer to implement by coalescing the block and primitive variable like this:

typedef void(^MyStoppableBlock)(MyBlock block, BOOL *stop);
- (void)primitive_cancellable_dispatch_after:(dispatch_time_t)delay block:(MyStoppableBlock)block;

Rather than the original:

- (void)primitive_cancellable_dispatch_after:(dispatch_time_t)delay block:(MyBlock)block stop:(BOOL*)stop;

this routine is never really used in any production apps, correct?

On iOS, yes.

macOS supports third-party non-app code, like daemons, and those can use dispatch_main. Indeed, this can even show up in App Store apps now that we have system extensions.

In regards, to ARC, you mentioned that it works hard to automate things, but it seems this is at the expense of giving up exact control.

Right. All automatic memory management systems trade convenience for control, and Objective-C ARC is no exception.

Is it a bug in the language that it’s not capturing the primitive BOOL variable?

Honestly, I’m not 100% sure. The rules for this sort of thing are complex, especially in Objective-C which inherits all of C’s pointer malarkey.

So, I was just curious as to where you were going with that or if I’m misunderstanding.

Yeah, sorry, I wasn’t clear here. C’s concurrency model doesn’t put BOOL, or any of the other ‘primitive’ types you mentioned, in a privileged position. It’s not safe to access any normal value concurrently from two different threads, regardless of the type of that value. So, I was contrasting BOOL against things that are safe between two different threads — like atomics, or a value protected by a lock — not against char.

The belief that primitive values are somehow privileged is a common misconception. I regularly see folks break these rules in scenarios very like yours. For example, they might have a background thread that periodically polls a stop global variable which is set by the main thread. They then sprinkle some volatile keywords around and hope it works. And lo! most of the time it does work, because C’s pointer rules mean that the compiler has to be pessimistic about its optimisations. However, the fact that it does work a lot of the time doesn’t make it valid C. Code that previously worked can fail when you change compiler, or change to a new architecture.

Anyway, the above explains why I jumped to the unsafe-threading-is-undefined conclusion )-:

Oh, speaking of architectures, be aware that Arm uses a different hardware memory model than Intel, and thus can cause confusion beyond the compiler. If you’re curious, check out the following article that just popped up in my Twitter feed…

https://research.swtch.com/hwmm


Oh, one final thing: The fact that you’re carrying a stop Boolean around suggests that you’re unaware of Dispatch work items [1]. Consider:

@import Foundation;

int main(int argc, char **argv) {
    #pragma unused(argc)
    #pragma unused(argv)
    dispatch_block_t __block block = dispatch_block_create(0, ^{
        while (YES) {
            NSLog(@"work item will sleep");
            [NSThread sleepForTimeInterval:0.3];
            if (dispatch_block_testcancel(block)) {
                NSLog(@"work item is cancelled");
                return;
            }
        }
    });
    dispatch_queue_t blockQueue = dispatch_queue_create("blockQueue", nil);

    NSLog(@"will schedule work item");
    dispatch_time_t sooner = dispatch_time(DISPATCH_TIME_NOW, 1 * NSEC_PER_SEC);
    dispatch_after(sooner, blockQueue, block);

    NSLog(@"will schedule canceller");
    dispatch_time_t later = dispatch_time(DISPATCH_TIME_NOW, 2 * NSEC_PER_SEC);
    dispatch_after(later, dispatch_get_main_queue(), ^{
        NSLog(@"canceller will cancel");
        dispatch_block_cancel(block);
    });

    dispatch_main();
}

which prints:

2021-06-30 13:08:38.215008+0100 xxot[57236:3897888] will schedule work item
2021-06-30 13:08:38.216201+0100 xxot[57236:3897888] will schedule canceller
2021-06-30 13:08:39.215953+0100 xxot[57236:3897920] work item will sleep
2021-06-30 13:08:39.519531+0100 xxot[57236:3897920] work item will sleep
2021-06-30 13:08:39.824799+0100 xxot[57236:3897920] work item will sleep
2021-06-30 13:08:40.130051+0100 xxot[57236:3897920] work item will sleep
2021-06-30 13:08:40.216644+0100 xxot[57236:3897935] canceller will cancel
2021-06-30 13:08:40.435300+0100 xxot[57236:3897920] work item is cancelled

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I’m using the Swift terminology here because it’s a lot less confusing. In Objective-C these are called Dispatch blocks, ’cause that’s not an overloaded term (-:

Variable not captured unless referencing explicitly from local assignment
 
 
Q