C10k and number of sockets

This topic, albeit very interesting, does not look like it was posted in the right location. This forum is focused on Apple API and specific Apple development questions. High level topics such as this will need to be discussed in another forum.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

is there no limitation on open sockets?

This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework [1].

I’m not aware of any hard limit to how many connections can be run by the Network framework. Feel free to test it and file a bug if you run into any inconvenient limits.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

[1] The one exception here is macOS, where Network framework has to run through the kernel in order to support NKEs. This is one of the reasons we’re in the process of phasing out NKE support, starting with there deprecation in the macOS 10.15 SDK.

what is the right Apple forum for a question: "what is the maximum number of open sockets on macOS?"

>This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework

Quinn, thanks, that's interesting.

Then maybe it is possible to create an alternative implementation of user-space socket API on top of Network framework, so that the socket API and apps that use it also benefit?

Obviusly i had macOS in mind, hard to imagine C10k server running on iPhone or apple watch 🙂

Then maybe it is possible to create an alternative implementation of user-space socket API on top of Network framework

This is not feasible, alas. BSD Sockets has to implemented in the kernel because sockets themselves are file descriptors, and file descriptors need to work in all sorts of kernel-y contexts (like in a call to

select

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

maybe that's a huge undertaking... but calls like "select" can be wrapped as well, i believe. normally user apps do not care (or even know) whether they are talking to a system version of read/select, etc or some wrapper library - provided the API exposed is the same.

calls like

select

can be wrapped as well, i believe.

I’ve seen people go down this path before. It does not end well.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

> This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework [1].

what are the underlying kernel/system primitives/APIs Network framework using? i assume the Network framework itself is not open source, is the underlying layer opensource?

The underlying infrastructure is

libnetwork

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework [1]. [1] The one exception here is macOS, where Network framework has to run through the kernel in order to support NKEs. This is one of the reasons we’re in the process of phasing out NKE support, starting with their deprecation in the macOS 10.15 SDK.

Hi Quinn, it's been a while since you wrote the above. Do you know if macOS is still an unfortunate exception that requires a socket per Network framework's connection? cc @eskimo