Max received datagram size in iOS packet tunnel?

I also got this error Error Domain=NSPOSIXErrorDomain Code=40 "Message too long" with NWUDPSession setReadHandler method on iOS 14.6, WiFi network.

Right, so as noted, you could try restricting an MTU to 1500 if you are using an IKEv2 protocol in your tunnel. Otherwise this may come down to flow control on the writes and reads that are going through your tunnel. For example, read so much data from the remote side, and then only write the data you have read. Likewise, do not read again until all of your writes go through.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

Thanks Matt.

I just want to add some more observe results:

On an iPhone 5s iOS 12.5.4 run the same application the Message too long doesn't happen no matter it's on WiFi or Cellular network;

On an iPhone 7 iOS 14.6, it's very easy to reproduce this issue on a WiFi network, but I never see it happen on a Cellular network;

I didn't change the MTU in the code but set the tunnelOverheadBytes instead. When it's set to be 44 or 0 the same issue happens;

If ignoring the EMSGSIZE error when it happens the setReadHandler seems to run into a Message too long error dead loop, so when it happens I have to restart the UDP session.

On an iPhone 7 iOS 14.6, it's very easy to reproduce this issue on a WiFi network, but I never see it happen on a Cellular network;

This is interesting as it may point to your flow control process not being the issue here. The reason why I say that is because when this is the case usually the Wi-Fi network ends up fine and the Cellular network breaks down and backs up the buffered data. Unless, of course, you have a very slow Wi-Fi network.

Another angle would be to check how much memory your iOS packet tunnel is consuming? The reason I ask is because there is a limit on how much memory a packet tunnel provider in iOS, last I checked it was ~15mb. Check that you are not hitting a ceiling here. If that checks out, then you may want to investigate your server side or custom packet encapsulation protocol that you are using, something could be going wrong here if messages are being written incorrectly.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

The Xcode shows the memory used by the NE is always lower than 5MB.

Excellent. That seems to checkout.

And I did a packet capture. When the Message too long error happens all the packets length are no more than 1500. I also logged the maximumDatagramLength of the UDP session, it's 1452 on the WiFi network. On the Cellular network it's 1402.

Thank you for this info. Knowing this, would it be possible to shave down the message size on the datagram's that you are sending to something that is well below both of these lengths you provided, just to see if that has an impact? Now, I understand that in the long run this may have to manage more reads/writes here, but as a test this would be a good data point to know.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com