Thanks for following up!
My request stems from the requirements of the MQTT protocol. According to the protocol, the client is expected to keep retrying the connection to the server until either the user explicitly stops it or a retry limit is reached.
In case B, where there’s a temporary issue like a "data center outage," the client should continue sending "Connect" requests until the server get recovered from "asteroid strike" (or until it hits the retry limit).
In scenario C, however, we know the connection is permanently defunct. In this case, there’s no value in endlessly retrying, as the "Connect" requests will not succeed. We'd prefer to prompt the user to establish a new connection rather than engaging in pointless retry attempts.
Post
Replies
Boosts
Views
Activity
Here’s an update on my findings in case anyone else is interested in this topic.
I attempted to keep the connection open while the app is in the background. As Quinn mentioned, when the app is backgrounded, the socket essentially "goes deaf." When I bring the app back to the foreground, the connection will encounters the ECONNABORTED error code from IO handlers.
My main issue is determining how to handle the connection after resuming the app. If the app was suspended but the connection is still viable, I was able to continue using the connection for further IO events after resume the app. However, if the connection is defunct, I have to recreate a new connection.
Unfortunately, I haven’t found a reliable way to distinguish between these two scenarios. For now, I’ve opted to always close and re-create the connection upon receiving an ECONNABORTED error, and I’m leaving it at that.
If anyone has insights on detecting when a socket resource has been reclaimed, I’d love to hear any suggestions for improvement!
Hi Quinn, sorry for revisiting this old question. Thank you so much for the explanation. I've been testing on Network Framework while app in background, and I have some further questions.
From TN2277 in section Data Socket:
If you do leave your data socket open when going into the background, you must correctly handle errors on that socket.
What is the error handling required by the system different from a "normal socket error handling"? The example given in the TN2772 was handling EBADF on socket reclaimed. Is there more information about what are the other errors we should take care of?
About suspension. The doc was quite vague about the
(app) becomes eligible for suspension when it stops doing the thing that prevented it from being suspend
I did more testing on this by creating a long running task which keep writing into the socket. It seems the application would be kept alive as long as the task is running. (It ran for about 3 hours before I stopped it. )
From my investigation, it seems that the Network Framework handler like nw_connection_set_state_changed_handler would be proceed as normal when the application in background. Does it means that the socket IO events would prevent the app from being suspended?
Also, probably because the socket was kept alive, I've never got error like EBADF in the tests above. Would you have any recommendation for a proper way to verify if the socket is affected by the suspension...?
Thank you so much for the detailed and patient explanation of my questions. It is really helpful and clarified a lot for me!
[quote='805638022, DTS Engineer, /thread/764555?answerId=805638022#805638022']
“… and then call some higher-level function to process that data” then you’re more likely to hit issues.
[/quote]
That's definitely my concern. I was trying to avoid the complication of dead locks, and went with the serial dispatch queue to so that the shared data are processed one at a time. Of cause, if locks are better solution here, I'm open to it.
I'm also looking into the target queue as you mentioned here, the target queue design sounds like similar to my current solution if I understand correctly. Please correct me if I was wrong.
Instead of putting everything on dispatch queue B,
we put user_data related block to the target queue so that they are always processed in order.
I was not able to edit the original post, therefore I will update in the replies.
I did some investigation and it looks like the completion handler block would not be guaranteed to run respect to any dispatch queue related order.
And here is my current solution to avoid any possible data racing issue: I always wrapped any user_data changes into the dispatch queue B. For example:
For example
static void send_message(){
dispatch_data_t data = dispatch_data_create(message, len(message), dispath_event_loop->dispatch_queue, DISPATCH_DATA_DESTRUCTOR_DEFAULT);
nw_connection_send(
nw_connection, data, NW_CONNECTION_DEFAULT_MESSAGE_CONTEXT, false, ^(nw_error_t error) {
dispatch_async(dispatch_queue_B, ^(){
user_data.client_status = SENT;
mem_release(user_data.message_to_sent);}
});
});
}
And so far it seems working. Feel free to correct me if I was wrong, and possibly share any better way to handle it.
Hi Quinn,
Thank you so much for the response. I have some follow-up questions about network suspension and resumption (or socket reclaim?).
From Networking and Multitasking
All iOS networking APIs are ultimately implemented in terms of the BSD Sockets API
From this statement, I got the impression that the Apple Network Framework uses BSD Sockets for its underlying implementation. However, in your reply from iOS/tvOS seem to reuse closed sockets, you mentioned
this is all Network framework and thus there are no sockets in play
I'm a little confused about what does the "no sockets" means here.
Here’s my understanding of the reclaim mechanism for Listening Sockets—please correct me if I'm wrong:
If the resource is reclaimed, to resume the app properly, the app needs to reconfigure the socket, perform the handshakes, and re-established the connection.
If the resource is not reclaimed yet, the socket is put in idle while the app is in the background. When the socket is resumed, the app can continue processing the requests. (Of course, as the socket was idled, the requests might fail or time out, and the app would need to handle these cases.)
One peculiar thing I’ve noticed is that the "reclaim" behavior seems inconsistent. While testing with the Network framework using the following steps from Networking and Multitasking
putting your app in the background
ensuring that the app is suspended
locking the screen
Sometimes when the socket is reclaimed, the Network framework sends out a TCP FIN packet and closes the connection, and other times it does not. Does this mean the behavior is undefined for reclaim, or am I doing something wrong?
Additionally, I observed the same results using my own BSD socket implementation and the Apple Network framework (the test app is based on the SWIFT-NIO library). It seems they behave similarly. As I planed to move to the Network Framework for TLS1.3 support, are there any features or behaviors that the Network Framework provides that I should be aware of?
Thanks again for your help!
Update: Resolved.
It turned out to be an issue for environment variable "HOME". "HOME" was set to a different value in terminal and failed the keychain operation.
Update: Resolved.
It turned out to be issue for my environment variable. The env var "HOME" was set to a different value in the terminal.
It was still failed. In fact the original issue was exactly SecKeychainCopyDefault failed when I run xcodebuild test.
However, if I ran default-keychain directly in the terminal, it can correctly return the default keychain. Therefore, I'm thinking if that is an issue with xcode setup on the CI host. Would I need to do a code-sign for xcode build?
Reading Resolving errSecInternalComponent errors during code signing, I also tried to run unlock keychain before run my test. As I'm running on Github CI, I dont have the password for the default keychain there. I tested by creating my own keychain like, but unfortunately it did not help.
security create-keychain -p pwd build.keychain
security default-keychain -s build.keychain
security unlock-keychain -p pwd build.keychain
Hi Quinn,
That's a great news and thank you so much!! I will try it out.
I followed the post here to contribute the key binary. (https://developer.apple.com/forums/thread/72445)
However, I could did something wrong with the raw keys. I will work on it.
Again, thank you so much.
Thank you so much for your help, otherwise I could not locate the issue. I really appreciated it!
I will see if I can figure it out.
I really appreciate your help here.
Here is the hex dump of a sample EC key in X963 format I'm using for test. Would this work?
unsigned char hard_code_key[] = {
0x04, 0x0F, 0x12, 0x9D, 0x04, 0x61, 0x7F, 0x4A, 0xB3, 0xC4, 0xE9, 0x99, 0xCF, 0xFF, 0x37, 0x04,
0x57, 0xA5, 0x7D, 0xFA, 0xC3, 0xE4, 0x2A, 0x25, 0xA9, 0xE4, 0x8D, 0x77, 0xF6, 0x54, 0x41, 0x8E,
0x40, 0x4A, 0x2C, 0x55, 0xF2, 0xAE, 0xDC, 0x37, 0xB2, 0x7F, 0x0B, 0x6A, 0x13, 0x00, 0xED, 0xF2,
0x1D, 0x19, 0xE4, 0x3E, 0xDA, 0x28, 0x41, 0x9C, 0xB2, 0x14, 0x18, 0xD4, 0x61, 0x10, 0xD1, 0x79,
0x61, 0x7B, 0x7B, 0xE9, 0x64, 0x38, 0x2C, 0x7E, 0x20, 0x88, 0x28, 0x09, 0x42, 0x9F, 0xCD, 0x51,
0x39, 0x91, 0x0A, 0x4F, 0xAF, 0x5B, 0xC3, 0xAD, 0xA9, 0x79, 0xE6, 0x87, 0xA9, 0x76, 0xEA, 0x13,
0xAC};
Though it really depends on the library user, I would say there is a high possibility that the library is used for a daemon context.
Regardless of that, as the previous implementation for RSA keys is using file-based keychain, I would like to keep it consistent and stick with the file-based keychain.
Could I import ECC key as file-based keychain? Would there be any attribute I need to take care for that?
Sorry, I was mean to say "client" only. I'm working on a library to help setup network connection with TLS, and testing with a TLS client build upon it.
I'm a little confused about the keychain implementation here.
As mentioned in On Mac Keychains:
The Keychain and SecKeychain APIs only talk to the file-based keychain. The SecItem API talks to either implementation. Specifically, it talks to the data protection keychain if you supply either the kSecUseDataProtectionKeychain or the kSecAttrSynchronizable attribute. If not, it talks to the file-based keychain.
The keychain should default to file-based, since I'm working with MacOS and I didn't set either kSecAttrSynchronizable or kSecUseDataProtectionKeychain. I didn't get why the keychain is considered as data protection here.
However, inspired by the article, I manually set the above two attributes to false. And eventually I got the error:
OSStatus -25304 : The specified item is no longer valid. It may have been deleted from the keychain.
It seems like I did something wrong when I create the SecKeyRef, so it failed to find it. I applied SecKeyCreateWithData to create the SecKey as described here Storing CryptoKit Keys in the Keychain. Though the example is for CryptoKit, I assume the API should also work for pure data... or am I wrong here?
Here is the implementation, and key_error returns NULL. I assume the key creation succeed.
CFMutableDictionaryRef parameters = CFDictionaryCreateMutable(my_alloc, 0, NULL, NULL);
CFDictionarySetValue(parameters, kSecAttrKeyType, kSecAttrKeyTypeECSECPrimeRandom);
CFDictionarySetValue(parameters, kSecAttrKeyClass, kSecAttrKeyClassPrivate);
CFDictionarySetValue(parameters, kSecUseDataProtectionKeychain, kCFBooleanFalse);
CFDictionarySetValue(parameters, kSecAttrSynchronizable, kCFBooleanFalse);
CFDictionarySetValue(parameters, kSecUseKeychain, keychain); // Not sure if the keychain attribute works here ...?
// key_data is the binary data read from ANSI file, which is a ECC key in X963 format.
SecKeyRef privKey = SecKeyCreateWithData(key_data, parameters, &key_error);