Network.framework trouble sending udp video frames ([] nw_protocol_ipv4_frame_output_finalizer [C3:2] Not enough bytes to fragment)

I have an app where we want to stream video frames locally from one device to another. We were initially using Multipeer, but wanted to force the user to have to be using infrastructure wifi rather than something like p2p BT.

Using Network.framework, I start a udp NWListener and send the host/port to the client which then connects. This all works well. On the app we compress the frames, send them to the other device, and decompress them on the other end. This all works fine except there are sometimes periods of freeze (oddly when the camera is trying to focus). There are no explicit errors from the framework, but we are getting this in the console during the freeze:

Code Block
2021-05-19 20:04:57.870871-0400 App Name[17116:5431473] [] nw_protocol_ipv4_frame_output_finalizer [C3:2] Not enough bytes to fragment 70806


Although we don't get an explicit error from Network.framework, I'm assuming that the message size is too big. I decided to just create an empty project and attempt to send one (uncompressed) frame across. I set it up just like in the WWDC 2018 video using bonjour and udp instead of host port and it got me:

Code Block 2021-05-20 15:42:59.656575-0400 NetworkTest[65379:1172081] [connection] nw_flow_prepare_output_frames [C1.2.1 fe80::9:9e05:f383:58a0%en10.52624@en10 ready socket-flow (satisfied (Path is satisfied), interface: en10, expensive)] Data won't fit in frame (9216 < 84272)
2021-05-20 15:42:59.656930-0400 NetworkTest[65379:1172081] [connection] nw_write_request_report [C1] Send failed with error "Message too long"
2021-05-20 15:42:59.657056-0400 NetworkTest[65379:1172081] [connection] nw_flow_prepare_output_frames [C1.2.1 fe80::9:9e05:f383:58a0%en10.52624@en10 ready socket-flow (satisfied (Path is satisfied), interface: en10, expensive)] Failed to use 1 frames, marking as failed
Failed to send data with error The operation couldn’t be completed. (Network.NWError error 0.)


So I guess my question is, in my first error is this a problem with the messages being too long? It seems like in general my message sizes are short enough but the occasional spike is causing a decent freeze, here is an output of printing data.count:


Code Block
22713
23313
23236
22278
22545
22342
22920
21996
70798
2021-05-19 20:04:57.870871-0400 Encore Artist Dev[17116:5431473] [] nw_protocol_ipv4_frame_output_finalizer [C3:2] Not enough bytes to fragment 70806


And for the second error in my test project, I was wondering why it is explicit and different. Perhaps because bonjour/udp instead of hostname/port/udp?

Finally, is there demonstration or something I could see that would help me learn how to split up these larger messages with Network.framework over udp? WWDC 2018 has a demo of doing exactly what I want to do, but they gloss over the splitting of frames parts and there's no source code that I could find. Thank you in advance!

This all works fine except there are sometimes periods of freeze

First up, I recommend that you check out Investigating Network Latency Problems. It’s my general advice on how to dig into issues like this.

I'm assuming that the message size is too big.

I wouldn’t assume anything here. Rather, use maximumDatagramSize to see if your packet will fit.

Finally, is there demonstration or something I could see that would
help me learn how to split up these larger messages with Network
framework over UDP?

There’s no ‘one size fits all’ approach to this. The usual strategy is to add a header that identifies the message and the segment within that message. The tricky part is in assigning the sizes to these fields, and especially the identifier. You want to keep it small, because every byte you use is pure overhead, but you also need it to be large enough to disambiguate.

Another tricky aspect is dealing with the reassembly. You don’t want to keep segments of incomplete messages around indefinitely, so you need to work out a strategy for discarding them. If the stream is real time — that is, if a message arrives too late you just discard it anyway — then you can use that deadline to discard these segments.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I wouldn’t assume anything here. Rather, use maximumDatagramSize to see if your packet will fit.

I'm a little confused about the number being returned here. As I send each frame, the majority are going through despite being significantly larger than this number. Here's a log where I log the data count I'm sending vs maximumDatagramSize. Only when there's a spike does the frame freeze and I see that error message.

Code Block
Sending size: 15344 - maximumDatagramSize reported: 1472
Sending size: 18182 - maximumDatagramSize reported: 1472
Sending size: 17145 - maximumDatagramSize reported: 1472
Sending size: 14539 - maximumDatagramSize reported: 1472
Sending size: 16288 - maximumDatagramSize reported: 1472
Sending size: 19660 - maximumDatagramSize reported: 1472
Sending size: 25691 - maximumDatagramSize reported: 1472
Sending size: 52437 - maximumDatagramSize reported: 1472
2021-05-24 08:49:08.983732-0400 App Name [23702:7182702] [] nw_protocol_ipv4_frame_output_finalizer [C3:2] Not enough bytes to fragment 52445

I'm unable to find any information on that particular error message and although I see it in the console, there is no NWError in the completion block of connection.send.


I'm a little confused about the number being returned here.

OK, lemme explain that…

The maximumDatagramSize property returns the maximum datagram size that fits in a single packet. A value of 1472 is pretty normal here; it’s the standard Ethernet-style packet size (1500) minus the IP header (20) minus the UDP header (8). If you send a datagram larger than this the system will have to fragment it.

IP fragmentation is a really bad option because the IP stack has no context to apply to the process; it always uses its standard best effort algorithm. This is bad in your case because:
  • It delays the delivery of data that might be useful, waiting for the fragmentation timeout.

  • It uses memory keeping fragments lying around long after they’re no longer relevant, because your real-time deadline has expired.

A real-time app should never use IP fragmentation. (Indeed even non-real-time apps should avoid IP fragmentation but that’s a whole different kettle of fish.)

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
Network.framework trouble sending udp video frames ([] nw_protocol_ipv4_frame_output_finalizer [C3:2] Not enough bytes to fragment)
 
 
Q