Network framework on macOS

App & System Services Networking Network

yetanotherme OP

Created May ’24

Replies 22

Boosts 1

Views 1.3k

Participants 3

This was mentioned in another thread 4 years ago:

This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework [1].

[1] The one exception here is macOS, where Network framework has to run through the kernel in order to support NKEs. This is one of the reasons we’re in the process of phasing out NKE support, starting with their deprecation in the macOS 10.15 SDK.

Is macOS still an unfortunate exception that requires a socket per Network framework's connection?

Boost

DTS Engineer OP

Apple

May ’24

I asked about this internally and one of my colleagues pointed out that the built-in firewall (System Settings > Network > Firewall) disables user-space networking. I just tried it here in my office and, yeah, my test tool starts using sockets as soon as I enable the firewall.

Note As with everything we’ve been talking about on this thread, this is an implementation detail that could well change in the future.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

yetanotherme OP

May ’24

I just tried it here in my office and, yeah, my test tool starts using sockets as soon as I enable the firewall.

On macOS 14 as well?

DTS Engineer OP

Apple

May ’24

Sorry, I should’ve mentioned, I was testing on macOS 14.4.1.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

yetanotherme OP

May ’24

I see, thank you!

Where does it leave us... There are a few things currently that could make user-space networking not working. Imagine you asking users to do a number of steps like switching the firewall off to make your app working – not many users will like doing that.

Note As with everything we’ve been talking about on this thread, this is an implementation detail that could well change in the future.

I hope it will!

DTS Engineer OP

Apple

Jun ’24

There are a few things currently that could make user-space networking not working. Imagine you asking users to do a number of steps

I’m not sure how the latter follows from the former. If your app uses a lot of network connections simultaneously, use setrlimit to raise the number of file descriptors it has access to (RLIMIT_NOFILE). See the setrlimit man page for details. If Network framework uses BSD Sockets, that’ll avoid this problem. And if Network framework uses user-space networking, the file descriptor limit isn’t relevant but there’s very little downside to increasing it.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

yetanotherme OP

Jun ’24

Ah, sorry, I must have misread your answer in the original thread! I thought you are implying that while there is a limitation of 64K on sockets (which translates to the corresponding limitation on the maximum simultaneous connections) these limitations is not applicable anymore now that we have user-space networking. The logical path I've been following was: as we do have situations still where user-space networking is inactive we thus still have the issue of 64K sockets -> connections! I must admit I assumed quite a lot from what you didn't say explicitly.

So instead of assuming any further: please confirm there is no issue of having more than 64K connections even in cases where user-space networking is not active. At worst I'd have to call setrlimit to bump the count beyond 64K (which is a non-privileged AppStore compatible call).

Thanks!

DTS Engineer OP

Apple

Jun ’24

I'd have to call setrlimit to bump the count beyond 64K (which is a non-privileged AppStore compatible call).

setrlimit is a supported API which you can call in any App Store app.

I’m not aware of any meaningful limit to how high you can raise this. However, if you go too far you’ll hit some other limit. For example, for open files, you’ll run out of inodes.

For sockets, you’ll hit other limits. At some point you’ll run the system out of mbufs. However, I think you’ll hit the NECP limit first [1]. That isn’t a simple count, but dependent on resource usage. It usually kicks in around 500-ish active flows.

This NECP limit applies to NW flows as well as sockets.

The symptoms of hitting the NECP limit is that things fail with otherwise hard to explain ENOSPC errors.

Taking a step back, if your goal is to run tens of thousands of network flows in a single process, I doubt you’ll be able to achieve that on Apple platforms. If that’s important to you, I recommend that you start by creating some prototypes. Make sure to:

Test that the flows work, not just that you can open them.
Test on all the platforms you’re targeting. You might, for example, see different results on iOS and macOS.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I discuss NECP in A Peek Behind the NECP Curtain, but I don’t go into details about its limits.