Problem with using a datagram socket in Network Extension for ICMP

Question

Created Sep ’19

Replies 4

Boosts 0

Participants 2

We have a case where we need to send, and receive, an ICMP packet from a Network Extension. Not *tunnel* an ICMP packet--that works fine--but send an ICMP packet from the NE that's not going through the tunnel.

The send is fine. I create a datagram socket with a protocol of IPPROTO_ICMP, and I can send the ICMP packet. I see it on the wire, I see the response on the wire.

When I call recvfrom() on the socket I get a permission error. I can see that there are 84 bytes waiting to be read, but I can't read them... Same fd I just sent from.

I tried binding the socket, but get a permission error when I try to bind(). I've tried bind() with address set to 0, INADDR_ANY, and to the local physical address, always a permission error.

Is the problem the lack of bind()? If so, does anyone have an idea about why bind() might be failing?

If the problem isn't lack of bind(), does anyone have an idea what could be happening here? Could it be a limitation on what can be done from a Network Extension?

Boost

Answer 1

kjbrock OP

Sep ’19

Ok, I've managed to get this working. I needed to do a connect() call. There had been some problems with that earlier. Now I'm seeing something really strange... I'm calling read() in instead of recvfrom(), since it's a connected socket, and some data appears to be getting altered in the packet before it's read off the socket.

The value of ip_len in the packet on the wire is 0x0054, or 84 bytes, which is correct.

The value of ip_len in the *received* packet, that is read off the socket, is 0x4000. I found an old (2008!) discussion on the net about this, and the consensus seemed to be that the length of the ip header (0x0016) had been subtracted and ip_len had then been converted to little-endian, giving us 0x4000.

This seems absurd... Why would this be done? Other than that one difference everything looks fine, but it'd be great if I could prevent this from happening at all rather than having to hack around it in IP parsing code...

0

Answer 2

DTS Engineer OP

Apple

Sep ’19

I needed to do a
connect
call.

I’m glad you found that. I suspect that the root cause of the permissions problem is the network interface access control that’s applied to tunnel providers to prevent packet loops, so it’s good to see that connecting the socket avoids the problem.

Why would this be done?

This is, IMO, a bug in the original BSD TCP/IP stack. All BSD-derived TCP/IP stacks maintain this bug for compatibility purposes.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

0

Answer 3

kjbrock OP

Sep ’19

Lovely :-( At least it's easy to work around once it's known

Oh, and yes, there is definitely a packet loop...

0

Answer 4

kjbrock OP

Dec ’19

We're running into another problem related to this that could be a catch-22 with the binding issue.

When we send an ICMP packet to 172.20.10.3, it gets sent out the wrong interface. There are two active interfaces on the system:

- 10.83.38.96/32

- 172.20.10.1/240

The packet gets sent out 10.83.38.96, even though it's destined for a local subnet.

I'm going to see if bind()-ing a specific address works for this, as opposed to bind()-ing with 0 and letting the system fill in the details. Is there any other way to force the packet to go out the correct interface? I'd expect it to be automatic, but I've got the packet traces to prove that wrong...

Kevin

0