What is the command to list all socket filters/extensions in use?

I am in the middle of investigating an issue arising in the call to setsockopt syscall where it returns an undocumented and unexpected errno.

What’s that value?

which seems to indicate there's no filters or extensions in play.

No third-party filters in play. TN3134 Network Extension provider deployment explains that a content filter must be packaged as a sysex on macOS, and thus no sysexes means there are no content filters. Or DNS proxies or packet filters. But there are other provider types that might be relevant and can be packaged as an appex, most notably, a transparent proxy.

Having said that, the netstat output you posted makes it clear that all the filters currently attached were attached by the OS. You are not dealing with third-party code here.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hello Quinn,

I am in the middle of investigating an issue arising in the call to setsockopt syscall where it returns an undocumented and unexpected errno.

What’s that value?

A IPv4 SOCK_DGRAM socket that's bound and subsequently a setsockopt on that socket for IP_ADD_MEMBERSHIP option is leading to the return value from that call to be -1 with errno set to 8, which gets reported as "Exec format error". The setsockopt reproducer is very trivial

#include <netinet/in.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <unistd.h>
#include <arpa/inet.h>

int main(int argc, char *argv[]) {
    if (argc != 3) {
        fprintf(stderr, "Error, expected usage: <program> <multicast-ip-address> <network-interface-ip-address>\n");
        fprintf(stderr, "example usage: ./a.out 225.4.5.6 192.168.1.2\n");
        return -1;
    }
    char *mcast_join_group_addr = argv[1];
    char *network_intf_addr = argv[2];
    fprintf(stderr, "test will join multicast group address = %s of network interface address = %s\n",
            mcast_join_group_addr, network_intf_addr);

    // create a datagram IPv4 socket
    int type = SOCK_DGRAM;
    int domain = AF_INET;
    int fd = socket(domain, type, 0);
    if (fd < 0) {
        fprintf(stderr, "FAILED to create socket, errno %d - %s\n", errno, strerror(errno));
        return -1;
    }
    fprintf(stderr, "SOCK_DGRAM socket created, fd=%d\n", fd);

    // bind the socket to a wildcard address and ephemeral port
    struct sockaddr_in sa;
    memset((char *) &sa, 0, sizeof(sa));
    sa.sin_family = AF_INET;
    sa.sin_port = 0;
    // bind to wildcard
    inet_pton(AF_INET, "0.0.0.0", &(sa.sin_addr.s_addr));
    socklen_t len = sizeof(sa);
    int b = bind(fd, (struct sockaddr *) &sa, len);
    if (b < 0) {
        fprintf(stderr, "failed to bind: errno=%d - %s\n", errno, strerror(errno));
        return -1;
    }
    fprintf(stderr, "socket successfully bound\n");

    // set IP_ADD_MEMBERSHIP socket option on the socket
    struct ip_mreq mreq;
    // multicast group address
    inet_pton(AF_INET, mcast_join_group_addr, &(mreq.imr_multiaddr.s_addr));
    // interface IP address
    inet_pton(AF_INET, network_intf_addr, &(mreq.imr_interface.s_addr));
    int opt = IP_ADD_MEMBERSHIP;
    void *optval = (void *) &mreq;
    int optlen = sizeof(mreq);

    fprintf(stderr, "setting IP_ADD_MEMBERSHIP on socket\n");
    int n = setsockopt(fd, IPPROTO_IP, opt, optval, optlen);
    if (n < 0) {
        fprintf(stderr, "FAILED - setsockopt(IP_ADD_MEMBERSHIP) returned %d with errno %d - %s\n",
                n, errno, strerror(errno));
        close(fd);
        return -1;
    }
    close(fd);
    fprintf(stderr, "SUCCESSFUL completion of the test\n");
}

The fact that the errno is set to (or atleast interpreted as a) ENOEXEC is surprising since man setsockopt makes no mention of that error for this call.

My guess is that some specific filter/extension code gets run through the setsockopt syscall. Reading through https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/NKEConceptual/socket_nke/socket_nke.html I suspected it could be some socket filter.

This issue has been reported to the JDK team since around a decade https://bugs.openjdk.org/browse/JDK-8144003 but it's only recently that we have started noticing it more frequently in our setups. It could be something to do with our macosx hosts, but at this point I don't have an idea of what tools/commands/options I should be using to understand what code from within setsockopt is interfering here.

Would you happen to know any tracing (ktrace?) that might help narrow this down further? The system logs (viewed through Console app) haven't shown anything specific.

Having said that, the netstat output you posted makes it clear that all the filters currently attached were attached by the OS. You are not dealing with third-party code here.

That's good to know. In context of multicasting (or more specifically that setsockopt IP_ADD_MEMBERSHIP option) do these OS attached filters play any role or apply any specific rules that I should be aware of?

A IPv4 SOCK_DGRAM socket that's bound and subsequently a setsockopt on that socket for IP_ADD_MEMBERSHIP option is leading to the return value from that call to be -1 with errno set to 8, which gets reported as "Exec format error".

To be clear, that’s most definitely a bug. I encourage you to file a bug report about it, even if you can’t reproduce it yourself. Ideally that bug report would include a sysdiagnose log taken by one of your users just after that reproduce it.

Please post your bug number, just for the record.

Now, I can’t guarantee that the solution to that bug might be that we add ENOEXEC to the man page, but someone from the networking team needs to make that call.

do these OS attached filters play any role or apply any specific rules that I should be aware of?

I can’t think of any good reason for them to trigger the behaviour you’re seeing, but it certainly possible that they might.

IMO the most likely candidate here is the built-in firewall. It’s a clear intersection between socket filters and executables. Two things:

• Is the uptick you mentioned correlated with macOS 15’s release? The firewall got a major rework in that release.

• Have you looked to see if reports of the problem are always from those folks using the firewall?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hello Quinn,

To be clear, that’s most definitely a bug. I encourage you to file a bug report about it, even if you can’t reproduce it yourself. Ideally that bug report would include a sysdiagnose log taken by one of your users just after that reproduce it.

Please post your bug number, just for the record.

I have now created a bug through feedback assistance. The id is FB15368430. I've attached the trivial .c code which reproduces this on several of the hosts that I have run this against. For now, I don't have access to sysdiagnose output. I will check if that can be shared from one of the hosts that we reproduce this issue on. I will upload to that issue once I get access to those logs.

Now, I can’t guarantee that the solution to that bug might be that we add ENOEXEC to the man page, but someone from the networking team needs to make that call.

Understood. In fact, more than the "man" page update, once the relevant team finds out the root cause of this issue, what would be useful is either details of what exactly causes this error and/or advice to application developers (those who call setsockopt) on what is expected of them to address this error or advice to network administrators on how to fix/updated their configurations to prevent this error.

For the record, during investigation of this issue, I've experimented with retrying the "setsockopt" when it fails with this errno, just to see if there is some kind of race or some such thing. But that hasn't helped - the subsequent call returns back with the same error.

Is the uptick you mentioned correlated with macOS 15’s release? The firewall got a major rework in that release.

I can answer this one for certain that this issue isn't related to macOS 15 release. None of the reports (including for hosts that run in our internal setups) have been against this version. In fact, we haven't yet started using this version in our setup.

Have you looked to see if reports of the problem are always from those folks using the firewall?

With help from admins who have access to some of these hosts, I know that some of the hosts on which this issue reproduces, has the firewall disabled. Specifically, on those hosts, "Settings" -> "Network" -> "Firewall" says "This computer's firewall is currently turned off. ...."

If there's anything else that you or others would like to know to narrow this down, please do let me know, either here on in the feedback issue FB15368430.