iOS 10 - 30 second delay incurred in many NSURLSessionDataTasks, associated with "SO_NOAPNFALLBK failed: [42] Protocol not available, dumping backtrace" logs

Hi all,

Since iOS 10, an internal app has started seeing many downloads slow to a crawl. Across perhaps 100 NSURLSessionData/Download tasks (ranging in size from 200kb to 5Mb) most download in < 1s, but a fair percentage take nearly exactly 30seconds from the NSURLSessionTask state going into .Running to .Completed. That already smells like some sort of timeout.

There are also a lot of those new networking logs, but as well as log noise, specific stack traces, like:

2016-09-25 22:57:22.238112 App[81553:15103629] [] __nwlog_err_simulate_crash simulate crash already simulated "nw_socket_set_common_sockopts setsockopt SO_NOAPNFALLBK failed: [42] Protocol not available"
2016-09-25 22:57:22.238305 App[81553:15103629] [] nw_socket_set_common_sockopts setsockopt SO_NOAPNFALLBK failed: [42] Protocol not available, dumping backtrace:
        [x86_64] libnetcore-856.1.8
    0  libsystem_network.dylib            0x00000001096bd80e __nw_create_backtrace_string + 123
    1  libnetwork.dylib                    0x000000010a180194 nw_socket_add_input_handler + 3002
    2  libnetwork.dylib                    0x000000010a15ddb8 nw_endpoint_flow_attach_protocols + 3768
    3  libnetwork.dylib                    0x000000010a15cdd5 nw_endpoint_flow_setup_socket + 563
    4  libnetwork.dylib                    0x000000010a15bb34 -[NWConcrete_nw_endpoint_flow startWithHandler:] + 2612
    5  libnetwork.dylib                    0x000000010a176d11 nw_endpoint_handler_path_change + 1261
    6  libnetwork.dylib                    0x000000010a176740 nw_endpoint_handler_start + 570
    7  libnetwork.dylib                    0x000000010a18e003 nw_endpoint_resolver_start_next_child + 2240
    8  libdispatch.dylib                  0x000000

If I were to hazard a guess, I'd say a background daemon has crashed, and it takes 30s for a watchdog to notice. I see two other people talking about this log in the forums, but all to do with JavaScript downloads fwiw: 101 Network error and here: Player on TVOS 10 stopped playing a lot of streams

There are more of these logs than slow downloads, but in the same order of magnitude (eg. for 60 or so 30s downloads there are 90 or so backtraces).

If the downloads continually failed I'd suspect a disallowed TLS protocol or something, but they all eventually succeed...

A few things of note with this connection - it's TLS, uses NTLM authentication (I know, I know) and also uses a client-side authentication certificate.

Next step is a DTS ticket I guess...