I finally found a time to experiment with Network.framework and I find the experience very pleasant. The API looks well thought out and is a pleasure to work with.
My app (on the App Store for around 13 years) uses UDP networking to create a mesh between multiple devices where each device acts as a server and client at the same time.
It does that by creating one UDP socket on each device bound to a *:<port> and then uses that socket to sendmsg to other peers. That way all peers can communicate with each other using their well known <port>. This all works great and fine.
To test the performance and verify the functionality I have multiple XCTestCase scenarios where I create multiple peers and simulate various communications and verify their correctness. Within the XCTestCase process that means creating multiple underlying sockets and then bind them to multiple local random <port>s. Works great.
Now I'm trying to port this functionality to Network.framework.
Let's assume two peers for now. Code simplified.
Prepare NWParameter instances for both peers, plain UDP for now.
// NOTE: params for peer 1
let p1 = NWParameters(dtls: nil, udp: .init())
p1.requiredLocalEndpoint = .hostPort(host: "0.0.0.0", port: 2222)
p1.allowLocalEndpointReuse = true
// NOTE: params for peer 2
let p2 = NWParameters(dtls: nil, udp: .init())
p2.requiredLocalEndpoint = .hostPort(host: "0.0.0.0", port: 3333)
p2.allowLocalEndpointReuse = true
Create NWListeners for each peer.
// NOTE: listener for peer 1 - callbacks omitted for brevity
let s1 = try NWListener(using: parameters)
s1.start(queue: DispatchQueue.main)
// NOTE: listener for peer 2 - callbacks omitted for brevity
let s2 = try NWListener(using: parameters)
s2.start(queue: DispatchQueue.main)
The listeners start correctly and I can verify that I have two UDP ports open on my machine and bound to port 2222 and 3333. I can use netcat -u to send UDP packets to them and correctly see the appropriate NWConnection objects being created and all callbacks invoked. So far so good.
Now in that XCTestCase, I want to exchange a packets between peer1 and peer2. So I will create appropriate NWConnection and send data.
// NOTE: connection to port 3333 from port 2222
let c1 = NWConnection(host: "127.0.0.1", port: 3333, using: p1)
c2.start(queue: DispatchQueue.main)
// NOTE: wait for the c1 state .ready
c2.send(content: ..., completion: ...)
And now comes the problem.
The connection transitions to .preparing state, with correct parameters, and then to .waiting state with Error 48.
[L1 ready, local endpoint: <NULL>, parameters: udp, local: 0.0.0.0:2222, definite, attribution: developer, server, port: 3333, path satisfied (Path is satisfied), interface: en0[802.11], ipv4, dns, uses wifi, service: <NULL>]
[L2 ready, local endpoint: <NULL>, parameters: udp, local: 0.0.0.0:3333, definite, attribution: developer, server, port: 2222, path satisfied (Path is satisfied), interface: en0[802.11], ipv4, dns, uses wifi, service: <NULL>]
nw_socket_connect [C1:1] connectx(6 (guarded), [srcif=0, srcaddr=0.0.0.0:2222, dstaddr=127.0.0.1:3333], SAE_ASSOCID_ANY, 0, NULL, 0, NULL, SAE_CONNID_ANY) failed: [48: Address already in use]
nw_socket_connect [C1:1] connectx failed (fd 6) [48: Address already in use]
nw_socket_connect connectx failed [48: Address already in use]
state: preparing connection: [C1 127.0.0.1:2222 udp, local: 0.0.0.0:2222, attribution: developer, path satisfied (Path is satisfied), interface: lo0]
state: waiting(POSIXErrorCode(rawValue: 48): Address already in use) connection: [C1 127.0.0.1:3333 udp, local: 0.0.0.0:2222, attribution: developer, path satisfied (Path is satisfied), interface: lo0]
I believe this happens because the connection c1 essentially tries to create under the hood a new socket or something instead of reusing the one prepared for s1.
I can make the c1 work if I create the NWConnection without binding to the same localEndpoint as the listener s1. But in that case the packets sent via c1 use random outgoing port.
What am I missing to make this scenario work in Network.framework ?
P.S. I was able to make it work using the following trick:
- bind the
s1NWListenerlocal endpoint to::2222(IPv6) - connect the
c1NWConnectionto127.0.0.1:3333(IPv4)
That way packets on the wire are sent/received correctly.
I would believe that this is a bug in the Network.framework. I can open a DTS if more information is needed.
