Hi there, I have some thread related questions regards to network framework completion callbacks. In short, how should I process cross thread data in the completion callbacks? Here are more details. I have a background serial dispatch queue (call it dispatch queue A) to sequentially process the nw_connection and any network io events. Meanwhile, user inputs are handled by serial dispatch queue ( dispatch queue B). How should I handle the cross thread user data in this case? (I write some simplified sample code below) struct { int client_status; char* message_to_sent; }user_data; nw_connection_t nw_connection; dispatch_queue_t dispatch_queue_A static void send_message(){ dispatch_data_t data = dispatch_data_create(message, len(message), dispath_event_loop->dispatch_queue, DISPATCH_DATA_DESTRUCTOR_DEFAULT); nw_connection_send( nw_connection, data, NW_CONNECTION_DEFAULT_MESSAGE_CONTEXT, false, ^(nw_error_t error) { user_data.client_status = SENT; mem_release(user_data.message_to_sent); }); }); } static void setup_connection(){ dispatch_queue_A= dispatch_queue_create("unique_id_a", DISPATCH_QUEUE_SERIAL); nw_connection = nw_connection_create(endpoint, params); nw_connection_set_state_changed_handler(){ if (state == nw_connection_state_ready) { user_data.client_status = CONNECTED } // ... other operations ... } nw_connection_start(nw_connection); nw_retain(nw_connection); } static void user_main(){ setup_connection() user_data.client_status = INIT; dispatch_queue_t dispatch_queue_B = dispatch_queue_create("unique_id_b", DISPATCH_QUEUE_SERIAL); // write socket dispatch_async(dispatch_queue_B, ^(){ if (user_data.client_status != CONNECTED ) return; user_data.message_to_sent = malloc(XX,***) // I would like to have all io events processed on dispatch queue A so that the io events would not interacted with the user events dispatch_async_f(dispatch_queue_A, send_message); // Disconnect block dispatch_async(dispatch_queue_B, ^(){ dispatch_async_f(dispatch_queue_A, ^(){ nw_connection_cancel(nw_connection) }); user_data.client_status = DISCONNECTING; }); // clean up connection and so on... } To be more specific, my questions would be: As I was using serial dispatch queue, I didn't protect the user_data here. However, which thread would the send_completion_handler get called? Would it be a data race condition where the Disconnect block and send_completion_handler both access user_data? If I protect the user_data with lock, it might block the thread. How does the dispatch queue make sure it would NOT put a related execution block onto the "blocked thread"?

Hi all, I'm developing an TCP socket SDK in C. The SDK is using Apple Network Framework and encountered some wired bad access issue occasionally on function nw_connection_send. Looking into the trace stack, it was bad access issue in nw_write_request_create, when it is trying to release a reference. However, I could not found more doc/source code details about nw_write_request_create. // on socket destroy, we will release the related nw_connection. increase_ref_count(socket) nw_connection_t nw_connection = socket->nw_connection; dispatch_data_t data = dispatch_data_create(message_ptr->ptr, message_ptr->len, dispath_event_loop, DISPATCH_DATA_DESTRUCTOR_FREE); // > Bad Access here < // While I check `nw_connection` and `data`, both seems available while the function get called. I tried to call dispatch_retain on `data`, but it was not helpful. nw_connection_send( nw_connection, data, NW_CONNECTION_DEFAULT_MESSAGE_CONTEXT, false, ^(nw_error_t error) { // process the message, we will release message_buf in this function. completed_fn(message_buf); reduce_ref_count(socket) } While I check nw_connection and data, both seems available while the function get called. I tried to call dispatch_retain on data, but it was not helpful. Is there any way to narrow down which object is releasing? As the issue happened occasionally (9 failure out of 10 attempts when I run multiple unit tests at the same time, and I rarely see it when I ran a single unit test). I would assume it was actually a race condition here. Is there a way to track down which object is released? I do understand it would be hard to track without knowing more design details of my SDK, but any related suggestions or ideas would be appreciated. Thanks in advance. More related source code: struct nw_socket{ nw_connection_t nw_connection; nw_parameters_t socket_options_to_params; dispatch_queue_t event_loop; // ... bunch of other parameters... struct ref_count ref_count; } static int s_socket_connect_fn( const struct socket_endpoint *remote_endpoint, struct dispatch_queue_t event_loop) { nw_socket = /*new socket memory allocation, increasing ref count*/ nw_endpoint_t endpoint = nw_endpoint_create_address(/* process remote_endpoint */); nw_socket->nw_connection = nw_connection_create(endpoint, nw_socket >socket_options_to_params); nw_release(endpoint); nw_socket->nw_connection->set_queue(nw_socket->nw_connection, event_loop); nw_socket->event_loop = event_loop; nw_connection_set_state_changed_handler(nw_socket->nw_connection, ^(nw_connection_state_t state, nw_error_t error) { // setup connection handler } nw_connection_start(nw_socket->nw_connection); nw_retain(nw_socket->nw_connection); } // nw_socket is ref counted, call the destroy function on ref_count reduced to 0 static void s_socket_impl_destroy(void *sock_ptr) { struct nw_socket *nw_socket = sock_ptr; /* Network Framework cleanup */ if (nw_socket->socket_options_to_params) { nw_release(nw_socket->socket_options_to_params); nw_socket->socket_options_to_params = NULL; } if (nw_socket->nw_connection) { nw_release(nw_socket->nw_connection); // Print here, to make sure the nw_connection was not released before nw_connection_send call. nw_socket->nw_connection = NULL; } // releasing memory and other parameters } static int s_socket_write_fn( struct nw_socket *socket, const struct bytePtr* message_ptr, // message_ptr is a pointer to allocated message_buf socket_on_write_completed_fn *completed_fn, void *message_buf) { // Ideally nw_connection would not be released, as socket ref_count is retained here. increase_ref_count(socket->ref_count); nw_connection_t nw_connection = socket->nw_connection; struct dispatch_queue_t dispatch_event_loop = socket->event_loop; dispatch_data_t data = dispatch_data_create(message_ptr->ptr, message_ptr->len, dispath_event_loop, DISPATCH_DATA_DESTRUCTOR_FREE); // > Bad Access here < // While I check `nw_connection` and `data`, both seems available while the function get called. I tried to call dispatch_retain on `data`, but it is not helpful. nw_connection_send( nw_connection, data, NW_CONNECTION_DEFAULT_MESSAGE_CONTEXT, false, ^(nw_error_t error) { // process the message, we will release message_buf in this function. completed_fn(message_buf); reduce_ref_count(socket) } }

Hi team, I'm working on an MQTT client for Apple platforms (macOS, iOS, and possibly tvOS and watchOS). I would like the client to listen to messages even when the application is in the background. I would appreciate any suggestions on the best approach to achieve this. Based on iOS Background Execution Limits, it seems that my best bet is to use a long-running background process with BGProcessingTaskRequest while setting up the connection. Does that sound like the right approach? Is there any limits for the bg tasks? I currently have a working BSD socket. I'm not sure if it is necessary to switch to the Network Framework to have the background task working, but I'm open to switching if it's necessary. If the approach works, does that mean I could built a http client to process large upload/download tasks without using NSURLSession? As I'm working on a cross platform project, it would be benefit if I dont need a separate http client implementation for Apple. Any insights on this topic would be greatly appreciated. Additionally, it's off topic, but the link to "WWDC 2020 Session 10063 Background Execution Demystified" (https://developer.apple.com/videos/play/wwdc2020/10063/) is broken. Is there a way to access the content there? Thanks in advance for your help and insights!

I'm having issue with keychain access for my SWIFT project. The keychain operations succeed while I run the test with Xcode.app (GUI), but failed when I run the test through command line tool xcodebuild. I assume I did something wrong with the environment. Is there any suggestion or instruction about how should I setup for the xcodebuild command line tool? Here is my unit test. static func run_shell(_ command: String) -> String { let task = Process() let pipe = Pipe() task.standardOutput = pipe task.standardError = pipe task.arguments = ["-c", command] task.launchPath = "/bin/zsh" task.standardInput = nil task.launch() let data = pipe.fileHandleForReading.readDataToEndOfFile() let output = String(data: data, encoding: .utf8)! return output } func testSecurityDefaultKeychain() throws { print(TLSContextTests.run_shell("security default-keychain")); } Other things I have tried: I got the same result if I use SecKeychainCopyDefault instead of the security command. If I directly run security command in my terminal, it worked fine. > security default-keychain "/Users/runner/Library/Keychains/login.keychain-db" I also tried with sudo xcodebuild & chmod a+x xcodebuild to make sure the tool has permission to access keychain, but it was not helpful. I had a post about the same issue a month ago. At that time I thought it was an issue for CI environment only. However, it turns out it was the xcodebuild. https://forums.developer.apple.com/forums/thread/747794

I'm setting up unit tests for my application using Xcode. When I tried to access default keychain on MacOS using SecKeychainCopyDefault, I got error OSStatus -25307, which means "A default keychain could not be found." The tests worked locally, and the issue only happened with Github Actions. Would anyone have any insight on this issue, or point me to some reading I can refer to? Thanks in advance! Here is some more tests I've done here. I tried to run "security default-keychain" on GithubAction, and I got &gt; security default-keychain "/Users/runner/Library/Keychains/login.keychain-db" However, when I tried to start a shell to run the same security command in my unit test, I got SecKeychainCopyDefault: A default keychain could not be found. Here is my test calling security command from shell: static func run_shell(_ command: String) -&gt; String { let task = Process() let pipe = Pipe() task.standardOutput = pipe task.standardError = pipe task.arguments = ["-c", command] task.launchPath = "/bin/zsh" task.standardInput = nil task.launch() let data = pipe.fileHandleForReading.readDataToEndOfFile() let output = String(data: data, encoding: .utf8)! return output } func testSecurityDefaultKeychain() throws { print(TLSContextTests.run_shell("security default-keychain")); }

Hi all, I'm working on a macOS project with C interface. And I'm trying to import my private key to a SecKeychainRef, but I always got error code -50. Would you have any advice or suggestion in it? Thanks in advance. Here is my code: // Get default keychain SecKeychainRef import_keychain = NULL; OSStatus keychain_status = SecKeychainCopyDefault(&import_keychain); // Create key from ECC key data in X963 format CFMutableDictionaryRef parameters = CFDictionaryCreateMutable(default_alloc, 0, NULL, NULL); CFDictionarySetValue(parameters, kSecAttrKeyType, kSecAttrKeyTypeECSECPrimeRandom); CFDictionarySetValue(parameters, kSecAttrKeyClass, kSecAttrKeyClassPrivate); CFDictionarySetValue(parameters, kSecAttrApplicationLabel, cfLabel); SecKeyRef private_key= SecKeyCreateWithData(hard_code_key_ref, parameters, &key_error); // Add seckey to key chain CFMutableDictionaryRef secItemParams = CFDictionaryCreateMutable(default_alloc, 0, NULL, NULL); CFDictionarySetValue(secItemParams, kSecClass, kSecClassKey); CFDictionarySetValue(secItemParams, kSecValueRef, privateKey); CFDictionarySetValue(secItemParams, kSecUseKeychain, import_keychain); OSStatus key_status = SecItemAdd(secItemParams, NULL); I also tried to test "SecItemAdd" with password value, but it also failed with -25308. I'm not sure if it is related or not. Here is the test code:             CFStringRef server = CFStringCreateWithCString(default_alloc, "example.com", kCFStringEncodingUTF8);             CFStringRef username = CFStringCreateWithCString(default_alloc, "username", kCFStringEncodingUTF8);             CFStringRef password = CFStringCreateWithCString(default_alloc, "password", kCFStringEncodingUTF8);             CFMutableDictionaryRef secItemParams = CFDictionaryCreateMutable(default_alloc, 0, NULL, NULL);             CFDictionarySetValue(secItemParams, kSecClass, kSecClassInternetPassword);             CFDictionarySetValue(secItemParams, kSecValueData, password);             CFDictionarySetValue(secItemParams, kSecAttrAccount, username);             CFDictionarySetValue(secItemParams, kSecAttrServer, server);                         CFDictionarySetValue(secItemParams, kSecUseKeychain, import_keychain);             CFDictionarySetValue(secItemParams, kSecAttrAccessible, kSecAttrAccessibleAlways); OSStatus key_status = SecItemAdd(secItemParams, NULL); The above code failed with "OSStatus -25308 : User interaction is not allowed." Any advice is welcomed. Thank you!