Crash on iOS 14.5：nw_endpoint_flow_copy_path + 44

App & System Services Core OS iOS Objective-C Network

Created Apr ’21

Replies 34

Boosts 0

Views 15k

Participants 20

Hi，
Apps crashed when enter foreground and want to connect with server，since iOS 14.5 released on April 27。

Crash stack like this：

Monitor Type: Mach Exception
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000010
Crashed Thread: 40
Pthread id: 867350
Thread 40 Crashed:
0 libnetwork.dylib nw_endpoint_flow_copy_path + 44
1 libnetwork.dylib nw_endpoint_flow_copy_path + 40
2 libnetwork.dylib nw_endpoint_flow_connected + 320
3 libnetwork.dylib nw_flow_connected + 3668
4 libnetwork.dylib nw_socket_connect + 584
5 libnetwork.dylib nw_endpoint_flow_connect + 164
6 libnetwork.dylib nw_endpoint_flow_setup_protocols + 3272
7 libnetwork.dylib -[NWConcrete_nw_endpoint_flow startWithHandler:] + 3616
8 libnetwork.dylib nw_endpoint_handler_path_change + 10592
9 libnetwork.dylib nw_endpoint_handler_start + 1184
10 libnetwork.dylib __nw_connection_start_block_invoke + 1460
11 imeituan __innerBlockOnce_block_invoke (SCRCrashMonitor_GCD.mm:191)
12 libdispatch.dylib _dispatch_call_block_and_release + 32
13 libdispatch.dylib _dispatch_client_callout + 20
14 libdispatch.dylib _dispatch_lane_serial_drain + 620
15 libdispatch.dylib _dispatch_lane_invoke + 456
16 libdispatch.dylib _dispatch_workloop_invoke + 1680
17 libdispatch.dylib _dispatch_workloop_worker_thread + 764
18 libsystem_pthread.dylib _pthread_wqthread + 276
Enqueue thread backtrace:
0 imeituan innerBlockOnce (SCRCrashMonitor_GCD.mm:0)
1 imeituan _dispatch_async (SCRCrashMonitor_GCD.mm:0)
2 libnetwork.dylib nw_connection_start + 280
3 libnetwork.dylib tcp_connection_start + 1496
4 CFNetwork 0x0000000180b7c3bc + 168
5 CFNetwork 0x0000000180b79b80 + 2776
6 CFNetwork 0x0000000180b77918 + 1724
7 CoreFoundation _CFStreamOpen + 140
8 imeituan -[GCDAsyncSocket openStreams] (GCDAsyncSocket.m:6998)
9 imeituan __29-[GCDAsyncSocket didConnect:]_block_invoke330 (GCDAsyncSocket.m:2369)
10 imeituan __29-[GCDAsyncSocket didConnect:]_block_invoke_2 (GCDAsyncSocket.m:2396)
11 imeituan __innerBlockOnce_block_invoke (SCRCrashMonitor_GCD.mm:191)
12 libdispatch.dylib _dispatch_call_block_and_release + 32
13 libdispatch.dylib _dispatch_client_callout + 20
14 libdispatch.dylib _dispatch_lane_serial_drain + 620
15 libdispatch.dylib _dispatch_lane_invoke + 404
16 libdispatch.dylib _dispatch_workloop_worker_thread + 764
17 libsystem_pthread.dylib _pthread_wqthread + 276
18 libsystem_pthread.dylib start_wqthread + 8

Boost

aakarshsasi OP

May ’21

Quick question: Are you using a third-party library for your networking? Perhaps GCDAsyncSocket?

Yes, we do

carrieni OP

May ’21

We use GCDAsyncSocket too.

DTS Engineer OP

Apple

May ’21

We use GCDAsyncSocket too.

OK. Let me start by being very clear about this: This is Apple’s bug to fix. GCDAsyncSocket is not doing anything wrong here, it just happens to be using the system in a way that triggers this bug.

Having said that, moving to Network framework would be a good workaround for this bug. There’s actually multiple reasons for this:

It works around this bug.
GCDAsyncSocket is based on APIs that are effectively deprecated [1].
In most cases GCDAsyncSocket users can migrate directly to Network framework without the need for any wrapper.
Network framework is the future of (non-HTTP) networking on our platforms. That’s where our efforts are being placed, and now that we’ve caught up with the basics this is likely to show more and more. As an example, folks using Network framework automatically get TLS 1.3 support; folks using older APIs, like CFSocketStream and BSD Sockets with Secure Transport, are stuck on TLS 1.2.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] It actually uses two different APIs, CFSocketStream and BSD Sockets. CFSocketStream is definitely on the path to formal deprecation. I can’t imagine us ever formally deprecating BSD Sockets but it is very much a legacy API.

carrieni OP

May ’21

Will it be fixed in 14.6 ？ @eskimo

DTS Engineer OP

Apple

May ’21

Will it be fixed in 14.6 ？

I am, alas, unable to predict the future.

I can say that it’s not fixed in the current 14.6rc (18F71). It also not fixed in the 14.7b1 (18G5023c) that we just started seeding. I recommend that you continue testing with 14.7 betas as they are released.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

K.H OP

May ’21

We have the same problem, This is a relatively serious problem by analyze our monitoring data. It caused a very large amount of crash and make user experience bad. If it is Apple’s bug, Why not fix it as soon as possible, moving to Network framework in a short time is not too easy for a system like base on GCDAsyncSocket.And there's not enough introduction to do it :(

ligaofeng OP

May ’21

Will it be fixed in 14.7 ？ @eskimo

Adithya Reddy OP

May ’21

@eskimo

Moving to Network.framework on such short notice is quite hard. iOS14.5 adoption only increasing.
Network.framework doesn't support iOS 11 (yes, we do have a chunk of users on iOS 11).

Please suggest if there's any other workaround here that Network team can think of.

DTS Engineer OP

Apple

May ’21

ligaofeng wrote:

Will it be fixed in 14.7 ？

I am, alas, unable to predict the future. I do, however, recommend that you continue to test with beta release systems as they become available.

Adithya Reddy wrote:

Moving to Network.framework on such short notice is quite hard.

Understood. I only suggested it because a) it will work around this problem, and b) it’s the right thing to do going forward.

Please suggest if there's any other workaround here

Not that I’m aware of. Well, there are other things you could (BSD Sockets, anyone?) but nothing that’s easier than moving to Network framework, which you’ve already ruled out.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

DTS Engineer OP

Apple

Jun ’21

We believe that the just-seeded iOS 14.7 beta 2 (18G5033e) has fixed the bug being discussed in this thread (r. 71651938). Please try it out, and let us know what you see.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

DTS Engineer OP

Apple

Jun ’21

if the ipv6 host is forcibly used for connection, is there any hope to reduce the occurrence of this crash?

I don’t know if the IP protocol version plays a part in this problem. Sorry.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

lion_liu@intsig.net OP

Jul ’21

Crashed: com.apple.network.connection-23

0 libnetwork.dylib 0x182fe8c90 nw_endpoint_flow_copy_path + 40

1 libnetwork.dylib 0x183007ab0 nw_endpoint_flow_connected + 344

2 libnetwork.dylib 0x1833ffd08 nw_flow_connected + 3732

3 libnetwork.dylib 0x183288708 nw_socket_connect + 564

4 libnetwork.dylib 0x182fe82ac nw_endpoint_flow_connect + 160

5 libnetwork.dylib 0x182fe3dbc nw_endpoint_flow_setup_protocols + 3300

6 libnetwork.dylib 0x182fdb69c -[NWConcrete_nw_endpoint_flow startWithHandler:] + 3644

7 libnetwork.dylib 0x1830e0dc0 nw_endpoint_handler_path_change + 10480

8 libnetwork.dylib 0x1830e8918 nw_endpoint_handler_start + 1172

9 libnetwork.dylib 0x183442c84 __nw_connection_start_block_invoke + 1448

10 libdispatch.dylib 0x1821a22b0 _dispatch_call_block_and_release + 24

11 libdispatch.dylib 0x1821a3298 _dispatch_client_callout + 16

12 libdispatch.dylib 0x18217f344 _dispatch_lane_serial_drain$VARIANT$armv81 + 568

13 libdispatch.dylib 0x18217fe60 _dispatch_lane_invoke$VARIANT$armv81 + 452

14 libdispatch.dylib 0x182180f70 _dispatch_workloop_invoke$VARIANT$armv81 + 1560

15 libdispatch.dylib 0x18218966c _dispatch_workloop_worker_thread + 692

16 libsystem_pthread.dylib 0x1ca9b95bc _pthread_wqthread + 272

17 libsystem_pthread.dylib 0x1ca9bc86c start_wqthread + 8

DTS Engineer OP

Apple

Jul ’21

can this fixed the bug ?

I’m not quite sure what you’re asking here, so here’s a recap:

This system bug seems to be triggered by GCDAsyncSocket. If you use Network framework directly, you won’t hit it.
Moreover, moving to Network framework is the right thing to do in the long term.
With regards Apple fixing this bug, see my post upthread.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

DTS Engineer OP

Apple

Jul ’21

We found that this problem still exists in the release version of iOS 14.7. So… Is this problem really fixed?

Well, that’s our understanding but it seems you have evidence that we are wrong. My advice is that you put that evidence into a bug report.

Please post your bug number, just for the record.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Adithya Reddy OP

Jul ’21

We are seeing occurrences of this crash in iOS v14.7.0 and v14.7.1 as well.

https://feedbackassistant.apple.com/feedback/9407987