MTU cache doesn't gets updated when PMTU is set.

HI,

I've created a virtual interface that used to get all outgoing packets, encapsulate with some VPN header, before resend them to their final destination through the physical adapter.

In order to choose the optimal MTU size that won't trigger fragmentation, I'd like to calculate the PMTU between the physical interface and the destination, subtracted by the encapsulation header size.

Then I'll set the virtual adapter's MTU with this result.

In order to do so, I poll the overall MTU cache using sysctl on macOS.

First, I verified that path mtu discovery is set

sysctl net.inet.tcp.path_mtu_discovery
net.inet.tcp.path_mtu_discovery: 1

Then, I tried to extract the cached pmtu for the gateway from the other size of the tunnel using the routing table .

static constexpr auto kSysctlMibLength = 6;

void get_pmtu_cache() {
    std::map<std::string, std::uint32_t> res;
    size_t size_needed = 0;
    std::vector<char> route_table;
    std::array<int, kSysctlMibLength> mib;
    char *next = nullptr;
    char *lim = nullptr;
    struct rt_msghdr *rtm = nullptr;
    struct sockaddr *saddr = nullptr;
    struct sockaddr_in *sockin = nullptr;
    char dest_ip_address[INET6_ADDRSTRLEN];
    
    mib[0] = CTL_NET;
    mib[1] = PF_ROUTE;
    mib[2] = 0;
    mib[3] = 0;
    mib[4] = NET_RT_DUMP;
    mib[5] = 0;

    // stage 1 : get the routing table 

    // get routing table size
    if (sysctl(mib.data(), kSysctlMibLength, nullptr, &size_needed, nullptr, 0) < 0) {
        return;
    }
    
    // allocate local var according to size 
    route_table.reserve(size_needed);
    
    // get routing table contents     
    if (sysctl(mib.data(), kSysctlMibLength, route_table.data(), &size_needed, nullptr, 0) < 0) {
        return;
    }

In the next step, I simple iterate the routing table elements and extract the following field for each destination : rt_msghdr.rt_metrics.rmx_mtu which is the path MTU from current endpoint to dest address.

    lim = route_table.data() + size_needed;
    for (next = route_table.data(); next < lim; next += rtm->rtm_msglen) {
        rtm = reinterpret_cast<struct rt_msghdr *>(next);
        
        saddr = reinterpret_cast<struct sockaddr *>(rtm + 1);
        if ((rtm->rtm_addrs & RTA_DST) != 0) { 
            sockin = reinterpret_cast<struct sockaddr_in *>(saddr);
            if (nullptr == inet_ntop(saddr->sa_family, &sockin->sin_addr.s_addr, dest_ip_address,INET6_ADDRSTRLEN)) {
                continue;
            }
            
            const std::string dest_ip_address_str(dest_ip_address, strlen(dest_ip_address));
            auto iter = res.find(dest_ip_address_str);

            if (iter == res.end() || iter->second > rtm->rtm_rmx.rmx_mtu) {
                res.insert_or_assign(dest_ip_address_str, rtm->rtm_rmx.rmx_mtu);
            }
        }
    }

when I finally print all the values in res I see that my pmtu to my VPN server is 1500, even-though I've set the server's mtu size to 1000, and I check that ping -D -s 1500 <server> doesn't work since packets from size 1500 that cannot be fragmanted won't work.

    auto item = res.find(vpn_server_address);
    if (item == res.cend()) {
      printf("no pmtu found to %s\n", vpn_server_address );
      return;
    }

    ret = item->second;

I've tried to trigger the pmtu discovery to the VPN server using nscurl by sending https requests there, but the pmtu still remained on 1500.

Perhaps I'm missing something ?

do I extract the pmtu correctly ? do I trigger the pmtu discovery by sending https messages using nscurl which is based on NSURLSession ?

Thanks !