TCP send() truncating data payload on a null byte

I'm experiencing a strange issue with sockets on OSX Sierra. If there is a null byte anywhere in my data, send() will truncate the data at the null byte and then append "\r\n" to my packet. However, send() still returns the length of the original packet I requested it to send, despite not actually sending that much data.


Here's some code:

int sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);

int ret = connect(sock, pAddrInfo->ai_addr, (int)pAddrInfo->ai_addrlen);
if(ret >= 0){
     char * packet = (char *)calloc(12,1);
     ((uint32_t *)packet)[0]= 0xffffffff;
     ((uint32_t *)packet)[1]= htonl(0x00111111);
     ((uint32_t *)packet)[2]= htonl(0x11110D0A);
     ret = send(sock, (void *)packet, 12, 0);
     if(ret < 0)
          printf("send failed");
     else
          printf("sent %d bytes", ret);
}else{
     printf("connect failed");
}


After this code I see that send returned 12 bytes were sent, but when capturing the packets with wireshark I see the TCP payload only has length 6 and that the data is FF FF FF FF 0D 0A. I expect to see FF FF FF FF 00 11 11 11 11 11 0D 0A.


This seems to only happen on OSX Sierra. I have tried on a machine running El Capitan and the code works as expected.

I am using xcode 8.0


Thanks.

Replies

I’m not sure what’s going on here but I will say that:

  • I can’t reproduce the problem

  • I suspect that the issue here is your testing methodology (but I could be wrong; the OS is big and complex and I’ve seen some wacky bugs in my time)

To reproduce the problem I put your code into a test tool (the full source is pasted in below) and ran it here in my office. Here’s what

tcpdump
shows:
… IP 192.168.1.104.49216 > 88.221.120.14.80: … length 12: HTTP
    0x0000:  04bf 6d8c c248 000c 29e6 b968 0800 4500
    0x0010:  0040 a745 4000 4006 0000 c0a8 0168 58dd
    0x0020:  780e c040 0050 597a f384 a827 ef3f 8018
    0x0030:  102c 932e 0000 0101 080a 3c86 ccfe b8fa
    0x0040:  0927 ffff ffff 0011 1111 1111 0d0a

As you can see, the expected values are all present and accounted for at the end of the packet.

The reason why I suspect a problem with your methodology are twofold:

  • If

    send
    was misbehaving like this in general, the OS simply wouldn’t work at all.
  • send
    is pretty much directly connected to the kernel. Architecturally, no one ‘below’
    send
    looks at the contents of the packet.

My advice:

  1. use my command line tool code to see if it still reproduces the problem

  2. double check your packet trace results

  3. try reproducing the problem on a fresh machine, just in case there’s something weird going on with your standard setup (like some crazy KEXT messing with your data)

Note I find VMs very helpful to satisfying this last point.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"
#include <sys/socket.h>
#include <netdb.h>
#include <stdlib.h>
#include <stdio.h>
#include <assert.h>

int main(int argc, char **argv) {
    #pragma unused(argc)
    #pragma unused(argv)
    struct addrinfo *  addrList;

    int err = getaddrinfo("www.apple.com", "80", NULL, &addrList);
    assert(err == 0);

    int sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); 

    int ret = connect(sock, addrList->ai_addr, addrList->ai_addrlen); 
    if(ret >= 0){ 
        char * packet = (char *)calloc(12,1); 
        ((uint32_t *)packet)[0]= 0xffffffff; 
        ((uint32_t *)packet)[1]= htonl(0x00111111); 
        ((uint32_t *)packet)[2]= htonl(0x11110D0A); 
        ssize_t bytesSent = send(sock, (void *)packet, 12, 0); 
        if(bytesSent < 0) 
              printf("send failed\n"); 
        else 
              printf("sent %zd bytes\n", bytesSent); 
        free(packet);
    }else{ 
        printf("connect failed\n"); 
    } 

    return EXIT_SUCCESS;
}

Using your code I am unable to reproduce the truncation issue, however I am now having a new issue where all data I send out on port 80 goes to the loopback address (127.0.0.1). If I use your code it will output sent 12 bytes regardless of the address I use. Even addresses that should not resolve, like 0.0.0.0, end up with connect() and send() returning success.


If I change the port to anything other than 80 the connect fails, as it should, and I see traffic with Wireshark. When the port is set to 80 I se no traffic on wireshark to the expected address, but I do see all of the tcp traffic, including the correct playload packet, sent to 127.0.0.1.


Using pfctl I cannot see any port forwarding rules that I would expect to cause this.


I am able to verify these behaviors on multiple machines running OS X Sierra, but not ony any machine running El Capitan.

Well, that’s weird. I recommend that you try this on a newly-installed 10.12, just to see if it’s caused by something that you happen to install on all your machines. I generally use a VM for this sort of thing.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"