APFS fast directory sizing API?

I do some manual directory sizing in my application, so I'm interested in using the new fast directory sizing capabilities on APFS volumes. The documentation currently says the following:


"The file system can enable fast directory sizing on empty directories. You cannot enable Fast Directory Sizing on directories containing files or other directories directly; you must instead first create a new directory, enable fast directory sizing on it, and then move the contents of the existing directory to the new directory."


My questions about this are:

  1. Is there any API that can tell me whether or not a given directory currently has fast directory sizing enabled for it? For directories that don't have it enabled, I would want to fall back to manually calculating the size in that case.
  2. Assuming I can tell fast directory sizing is enabled, what is the API for actually retrieving the calculated size? Can you just look at the value returned for the NSFileSize key after retrieving file attributes with NSFileManager? Or is there some new API for this?

Replies

Some new constants—ATTR_DIR_ALLOCSIZE, ATTR_DIR_IOBLOCKSIZE, and ATTR_DIR_DATALENGTH—recently showed up in the man page for getattrlist(2) (starting in 10.12.4), so I assume those are what are what we should use to retrieve those values once FDS is turned on for a folder.


How to actually turn it on is something I haven't figured out yet.

Thanks for that tip, I'll have to play around with those and see what they produce. I might go ahead and file a documentation bug, since this information is obviously very lacking at the moment.

Hey, do you have any update? I'm interested too 🙂


From man getattrlist:

ATTR_DIR_ALLOCSIZE An off_t containing the number of bytes on disk used by the directory (the physical size).

ATTR_DIR_IOBLOCKSIZE A u_int32_t containing the optimal block size when reading or writing data.

ATTR_DIR_DATALENGTH An off_t containing the length of the directory in bytes (the logical size).


I scanned an apfs startup disk and it seems that ATTR_DIR_ALLOCSIZE is always 0, ATTR_DIR_IOBLOCKSIZE is always 4MB and ATTR_DIR_DATALENGTH is some small amount around 200 bytes, obviously the size of the records, not actual file data.


How do we turn on the FDS, programmatically with an API and with OS tools?

Yeah, the documentation suggests it needs to be turned on for a directory in order to work:


https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/Features/Features.html

Fast Directory Sizing

Fast directory sizing allows Apple File System to quickly compute the total space used by a directory hierarchy, and update it as the hierarchy evolves.

Fast directory sizing works by precomputing the size of directory as content is added and removed. Therefore, it is most appropriate for directories that contain many files and have relatively little churn. For example, a user’s Documents folder is a good candidate for fast directory sizing, whereas the

/tmp
directory would not.

The file system can enable fast directory sizing on empty directories. You cannot enable Fast Directory Sizing on directories containing files or other directories directly; you must instead first create a new directory, enable fast directory sizing on it, and then move the contents of the existing directory to the new directory.

No idea how to actually turn it on, though. You'd think there'd be something in setattrlist, but I'm just not seeing it.

I'd love to just be able to query if any directories have this on and warm / remember to not perform high churn operations there. So far, best as I can tell, this is not enabled anywhere on APFS and is a theoretical benefit and not something that's enabled or rolled out yet. Does anyone know of a directory where this is on by default?