What kind of file descriptors are counted in for ulimit/rlimit?

Question

Created Apr ’23

Replies 1

Boosts 0

Participants 2

As a Developer, I want to provide a tool for sysadmins to determine how many files are open in the system. This is useful when combined with the information about resource limits (or ulimit) for processes. The lsof utility lists different types of open files, e.g. regular files and txt files. I have read the lsof source code extensively and would really like to know the following:

1.) Which open file descriptor types are counted by the kernel as part of the resource limit imposed on maximum open files. In other words: which file types as reported by lsof are counted in the open file statistics, such that opening more of them after reaching the resource limit would result in an error.

1.1.) Are txt files counted for the ulimit/resource limit?

2.) I don't quite understand what txt files are exactly. The manual says "program code and data", but what data exactly? I see things like:

ULimitTes 20517 federico  txt (...) &#x2F;System&#x2F;Library&#x2F;Fonts&#x2F;Apple Color Emoji.ttc

being reported as a txt file. If I, however, open a file within a dynamic library (with fopen), that is then linked into a program and run, that file is reported with a corresponding file descriptor on lsof, e.g.:

ULimitTes 20517 federico    6r   REG (...) &#x2F;Users&#x2F;federico&#x2F;INPUT

I expected that file to be a txt file, given that it wasn't opened by the program itself but by a library used by the program. Any hints to further clarify the nature of txt files would be appreciated!

3.) If I setrlimit(RLIMIT_NPROC) to 2 in a program but attempt to open 3 files with fopen, this succeeds. I would have expected the third file open operation to fail, as the limit was set to 2. So what does the limit set by setrlimit(RLIMIT_NPROC) (and then verified by querying getrlimit(RLIMIT_NPROC)) do?

Many thanks in advance for any hints on this! :-)

Boost

Answer 1

DTS Engineer OP

Apple

Apr ’23

Which open file descriptor types are counted by the kernel as part of the resource limit imposed on maximum open files.

The RLIMIT_NOFILE limit is on file descriptors. It’s possible to access a file without using a file descriptor by memory mapping (see the code snippet below). When you do that, lsof reports the file as txt:

% # In one Terminal window...
% .&#x2F;Test728832
will pause, pid: 39239
^C
% # In a different Terminal window..
% lsof -p 39239
…
Test72883 39239 quinn  txt  REG  1,5  19 169603546 &#x2F;Users&#x2F;quinn&#x2F;test.txt
…

The txt is short for text, which makes sense when you realise that:

Unix-y systems use the term text to refer to code. For example, the code segment within a Mach-O executable is called __TEXT.
Way back in the day, memory mapping was only used for executable code. mmap is a ‘recent’ addition (-:

If I setrlimit(RLIMIT_NPROC) to 2 in a program but attempt to open 3 files with fopen, this succeeds.

Yep. Because the correct resource limit for file descriptors is RLIMIT_NOFILE.

ps If you’re going to monkey around with the low-level aspects of macOS, I encourage you to grab some reference books on that topic. The two that I specifically recommend for this task are:

Advanced Programming in the Unix Environment, by Stevens [1]
Mac OS X Internals: A Systems Approach, by Singh

The latter is way out of date now but there’s no other resource that covers macOS as well.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Well, my version is by Stevens but the latest edition was updated by Rago. I haven’t read that version but I’ve read other stuff by Rago and he’s a worthy successor to Stevens IMO (-:

import Foundation

func main() {
    let fd = open("&#x2F;Users&#x2F;quinn&#x2F;test.txt", O_RDONLY)
    assert(fd >= 0)

    var sb = stat()
    let success = fstat(fd, &amp;sb) >= 0
    assert(success)

    let a = mmap(nil, Int(sb.st_size), PROT_READ, MAP_FILE | MAP_PRIVATE, fd, 0)
    assert(a != MAP_FAILED)
    
    let junk = close(fd)
    assert(junk >= 0)
    
    print("will pause, pid: \(getpid())")
    fflush(stdout)
    pause()
}

main()

0