where is macOS APIs including user space file system support?

Hi there,

From "Platforms State of the Union" Video macOS section I know macOS has new API of user space file system

and iPhone mirroring, and delivers new APIs including user space file system support and major improvements to MapKit.

But I lookup the API diff, I don't find any added API. Where can I find the user space file system API ? I really want to develop an APP which need user space file system API.

Platforms State of the Union Video corresponding timeline detail: https://youtu.be/YJZ5YcMsgD4?t=3153

FSKit but it’s not documented yet and there’s no sessions about it.

There is some framework documentation available at https://developer.apple.com/documentation/fskit?language=objc

@yalarmorghulis I would like to better understand your use case. What do you need from a userspace filesystem API?

Great question OP! I noticed the same info in Platforms SOTU and was looking for a session last night as well.

I would love to see a well documented API that allows for clear and straightforward way to grant permissions to enable apps read/write to specific folders.

It is not really clear if one wants to develop a file-management utility how to go about it unless the app is not sandboxed. And in that case can it even get app store approval? What would be the process?

The straightforward ways to access files in a sandboxed approach are basically unworkable for any practical utility apps. The sandbox folder is nested deep under $HOME/Library, and $HOME/Documents/ is not sufficient. e.g. One might want to access an SSD.

I would love to see a way to carve out a writeable space on the Mac volume, or on an external volume as $EXTVOLUME or $HOME/<AppName> with read access to non-system folders and pretty much any media under $HOME. Or provide us a clear path to certifying non-sandboxed apps.

Great question OP! I noticed the same info in Platforms SOTU and was looking for a session last night as well.

Just to be clear, FSKit is used to add support for new volume formats to the mac. It does not provide any access to the broader file system or address any of the issues you've raised.

I would love to see a well documented API that allows for clear and straightforward way to grant permissions to enable apps read/write to specific folders.

The most direct solution here is to present an open panel to the user and have them select the Folder(s) your app should have access to. You can the preserve access to that directory using a security scoped bookmark, giving your app access to the same directory in the future. Note that this technique can be used to grant and preserver your app's to any file/directory the user has access to.

It is not really clear if one wants to develop a file-management utility how to go about it unless the app is not sandboxed.

What are you trying to do and how is the sandbox interfering with it?

And in that case can it even get app store approval?

The only issue the approach above can't handle is file access that requires escalated privileges, but those apps are not allowed on the App Store.


Kevin Elliott
DTS Engineer, CoreOS/Hardware

@Systems Engineer Thank you for your reply and the API. I don't know if fskit can solve my scenario (sorry I don't figure out the detail of fskit, can you add example code to demonstrate how these API work together?)
I've two idea and want to see if user space file system can achieve it.
The first one is I want to develop an app which can mount cloud storage (AWS S3 etc) on my macOS as file system directory, user can save data to the directory, and I can use end to end encrypt to protect user data then save it to cloud storage.
The second one is I want filesystem has ability to quick revert file/folder to history version (like git, docker layer filesystem or APFS snapshot, but APFS snapshot is the whole volume). with this ability, my app can batch process media like audio/video with multi steps, if something is not what I want, I can quick revert to history version.

Thanks for the reply, and advice about open panel and security scoped bookmark. I am keen to try it out and see if those suggestions work.

What are you trying to do and how is the sandbox interfering with it?

I want to either organize in place (move and rename) files and folders under a particular directory tree. Probably a sub-folder or the users home. ~/somedir/ with access to all of the subdir items without needing to add those individually. Or scan ~/somedir/ and import to another folder tree ~/outputdir. It could be an external volume ssd.

Sandbox by default wouldn't allow that using nsfilemanager iterators after the SwiftUI folder selection dialogue.

With sandbox enabled I have to copy everything under "Downloads" and run the code. Not ideal, so I set entitlements to not sandboxed and put in lots of checks for system folders etc to lock it down.

Hoping to come up with a flexible solution that can be released to the app store.

I want to either organize in place (move and rename) files and folders under a particular directory tree. Probably a sub-folder or the users home. ~/somedir/ with access to all of the subdir items without needing to add those individually. Or scan ~/somedir/ and import to another folder tree ~/outputdir. It could be an external volume ssd.

Sandbox by default wouldn't allow that using nsfilemanager iterators after the SwiftUI folder selection dialogue.

I haven't really looked at the Swift UI APIs for this, but what you're describing works fine with NSOpenPanel. I was actually worried something had changed or I'd gone crazy, so I ended up modifying test project I had laying around JUST to be sure everything worked as I was expecting. See code below.

DISCLAIMER: The code below is greatly simplified test code and shouldn't really be used in any real project. Also, this code is ObjectiveC because it's Friday of WWDC week and that happened to be the starting project I had at hand.

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
    // Insert code here to initialize your application
    NSOpenPanel *myOpenPanel = [NSOpenPanel openPanel];
    [myOpenPanel setAllowsMultipleSelection: NO];
    myOpenPanel.canChooseDirectories = YES;
    [myOpenPanel runModal];
    NSURL* fileURL= myOpenPanel.URL;
    
    NSFileManager* fileMan = [NSFileManager defaultManager];
    NSURL* newFileURL = [fileURL URLByAppendingPathComponent: @"NewFile.txt"];
    NSURL*  dirURL = [fileURL URLByAppendingPathComponent: @"MyFolder"];
    NSString* junkString = @"Nothing to see here";

    BOOL result = true;
    result = [fileMan removeItemAtURL: dirURL error: NULL];
    NSLog(@"RemoveDirResult %d", result);
#if 1 //Write TestFile
    result = [fileMan removeItemAtURL: newFileURL error: NULL];
    NSLog(@"RemoveResult %d", result);
    result = [junkString writeToURL: newFileURL atomically: YES encoding: NSUTF8StringEncoding error: NULL];
    NSLog(@"WriteResult %d", result);
#endif
    
    NSString* readString = [NSString stringWithContentsOfURL: newFileURL encoding: NSUTF8StringEncoding error: NULL];
    NSLog(@"readString= %@", readString);
    if(result)
    {
        for (int i = 0; i < 10; i++) {
            result = [fileMan createDirectoryAtURL: dirURL withIntermediateDirectories:NO attributes: NULL error:NULL];
            NSLog(@"createDirectoryAtURL %d", result);
            newFileURL = [dirURL URLByAppendingPathComponent: @"Buriedfile"];
            result = [junkString writeToURL: newFileURL atomically: YES encoding: NSUTF8StringEncoding error: NULL];
            if(!result)
            {
                NSLog(@"createDirectoryAtURL %d %@", result, dirURL.path);
                break;
            }
            dirURL = [dirURL URLByAppendingPathComponent: @"MyFolder"];
        }
    }
}

If you share your SwfitUI code I can take a look, however, from broad experience most issues in this area are caused by one of two things:

  1. Switching over to string paths instead of using the URL object. Your app's access to the file is tied to the URL object you receive from the system, so you can't let the object be destroyed until you've either finished your work or used a security scoped bookmark to preserve access. More broadly, using string paths to represent files is a Bad Idea™.

  2. Inadvertently manipulating the NSURL in ways that don't actually make sense. Because NSURL is used for both network and file paths, it's easy to write code that appears reasonable but is actually quite odd. Notably, using any of the "...WithString:" URL initializers to create a path is simply wrong, as it initializes the file URL by parsing the path using network RFC. Depending on the input string it will either work fine or completely distort the path.


Kevin Elliott
DTS Engineer, CoreOS/Hardware

where is macOS APIs including user space file system support?
 
 
Q