I believe the fix @kithrup mentioned has problems in some cases. The behavior I see is:
All working fine and I have my TransparentProxy provider handling the all network connections (it is setup with a catch all rule).
I see errors(see below) in unified log from kernel.
All connections die in TransparentProxy, I see "write error: Flow is not connected" for all existing connections it was maintaining.
No new "handleNewFlow" method calls, my networking is down and I can't connect to anything.
I "sudo kill -9 " and after about 10 seconds my networking is working again.
The errors from the kernel are:
2024-05-12 13:11:03.745695+0300 0x3e63b3 Error 0x0 0 0 kernel: (250387962): mbuf_split failed: 12
2024-05-12 13:11:03.746038+0300 0x3e63b3 Error 0x0 30596 0 com.my.networkextension: (libnetworkextension.dylib) [com.apple.networkextension:] (0): data: write error: Cannot allocate memory
2024-05-12 13:11:03.746176+0300 0x3e63b3 Default 0x0 30596 0 com.my.networkextension: (libnetworkextension.dylib) [com.apple.networkextension:] (0): Aborting the director
I found that this "mbuf_split failed:" is from flow divert:
https://opensource.apple.com/source/xnu/xnu-4570.1.46/bsd/netinet/flow_divert.c.auto.html
flow_divert_send_app_data() function.
It looks like @kithrup mentioned that it tries to split the buffer but the split fails and then it kills something inside the extension(closes divert socket?) rendering networking completely dead until extension restart.
I cannot reproduce this consistently but it happens quite a lot in our product and we are scratching our heads trying to find a solution.