Sanboxed apps won't open 3rd party filesystem files

Hi,

What is required for a 3rd party filesystem kext to integrate with sandboxed apps?

Any pointers and/or assistance would be greatly appreciated.

Unfortunately, I don't think "sandboxed apps" are what's actually broken, which makes this harder to answer. The file system itself isn't (directly) involved with that sandbox at all, so it doesn't require anything "special" to support it.

What's more likely here is that there's an issue with your file system's implementation that's breaking one of the system expectation and the sandbox system just happens to be the failure point you've noticed. What EXACTLY that would be is very hard to guess that, however, the first thing I would check is whether your inode values are stable (two process retrieving the same file both receive the same inode value). Bookmarks (both "regular" and security scoped) will break if the inode value isn't stable.

-Kevin

Thanks for replying. I've run stat(1) asynchronously to verify that two processes querying the same file residing on my filesystem get the identical inode value. See below.

% stat -f 'inode: %i' t.txt&; stat -f 'inode: %i' t.txt&
[1] 1978
[2] 1979

inode: 1954135111

[2]  + done       stat -f 'inode: %i' t.txt

inode: 1954135111

[1]  + done       stat -f 'inode: %i' t.txt

What are the system expectations of a filesystem that my implementation should meet? Is it some struct vfs_attr value that should be set? It has to be something specific, shouldn't it? Is there anything else you could recommend in terms of troubleshooting this? Thanks.

When launching Preview.app on the command line, I get an 'Operation not permitted' error returned.

% /System/Applications/Preview.app/Contents/MacOS/Preview /Volumes/myfs/Through_the_pines.jpg
2024-06-17 14:23:16.826 Preview[2745:37915] PVImageContainer initWithURL:file:///Volumes/myfs/Through_the_pines.jpg failed, error = Error Domain=NSCocoaErrorDomain Code=257 "The file “Through_the_pines.jpg” couldn’t be opened because you don’t have permission to view it." UserInfo={NSFilePath=/Volumes/myfs/Through_the_pines.jpg, NSUnderlyingError=0x6000022302a0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}

File permissions allow reading for all users:

% ls -l /Volumes/myfs/Through_the_pines.jpg 
-rw-rw-r--@ 1 developer  staff  2396454 Mar 25 10:48 /Volumes/myfs/Through_the_pines.jpg

Am I right in assuming it's sandbox related?

Am I right in assuming it's sandbox related?

Maybe, but there are other issue as well. Starting here:

When launching Preview.app on the command line, I get an 'Operation not permitted' error returned.

/System/Applications/Preview.app/Contents/MacOS/Preview /Volumes/myfs/Through_the_pines.jpg

This is not a valid way to open a file with an application. Two different issues:

1. Launch arguments are not the standard mechanism for delivering document into applications. There is still code in NSApplication that parses command line arguments and converts them into standard open commands, but it's not a mechanism I'd recommend relying on or really consider "valid".

2. Attempting to directly launch an app through it's executable is not the standard launch operation and can cause other problems later in the target apps lifetime. In the case of most system apps (including Preview.app) attempting to do so will immediately crash the target process with a log that includes this:

Exception Type:  EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: CODESIGNING 4 Launch Constraint Violation

My guess is that you avoided that crash because you're running on a machine where SIP was disabled, however, I think that would also have disabled most of the relevant sandbox protections.

SO, starting the conversation over a bit, this is the correct way to launch an app from terminal is the "open" command. For your particular example, that command would be:

open -a /System/Applications/Preview.app/  /Volumes/myfs/Through_the_pines.jpg

You could also do:

open /Volumes/myfs/Through_the_pines.jpg

...which would be the same as double clicking on the file in the Finder.

Am I right in assuming it's sandbox related?

If Terminal.app has access to a given file, then a valid "open" command should open that file in the target app. Note that "access" here means both POSIX and sandbox permission, so you'd want to read the file with Terminal to confirm it has access.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks very much for clarifying the proper use of open(1) and for being willing to help.

Now, when launching Preview.app through 'open(1) -a' or by double clicking the image file in Finder, Preview.app launches:

% open -a /System/Applications/Preview.app/Contents/MacOS/Preview /tmp/myfs/Through_the_pines.jpg

lsof(8) reports the image file as being opened by Preview.app:

% lsof -p `pgrep -xi preview` | grep Through
Preview 2825 developer    3r   REG   54,2  2396454          3001938654 /private/tmp/myfs/Through_the_pines.jpg

But the contents of the image file aren't being displayed.

System logs have the following records:

2024-06-20 09:38:25.821809+0300 0x548e     Default     0x0                  2823   2    open: (LaunchServices) [com.apple.launchservices:open] Opening document <FSNode 0x600003f790a0> { isDir = n, path = '/private/tmp/myfs/Through_the_pines.jpg' } with application <FSNode 0x600003f79160> { isDir = y, path = '/System/Applications/Preview.app' }
2024-06-20 09:38:26.094972+0300 0x54da     Default     0x0                  2825   0    Preview: (Foundation) [com.apple.foundation.filecoordination:claims] Read options: 0 -- URL: file:///private/tmp/myfs/Through_the_pines.jpg -- purposeID: 78FD7C2A-102A-4679-A0FE-4854596E124C -- claimID: 2D45E1F1-8533-4F26-ADE7-22E92590C39A

The same behaviour is observed with pdf files.

As my filesystem is a network one, I thought I'd go and see if https://github.com/apple-oss-distributions/SMBClient offers anythings useful, as opening image and/or pdf files works there.

In the meantime, if you have any more tips for me, it would be much appreciated.

Is there a specific VFS attribute that meets the system's expectation of a filesystem in terms of sandbox(7) security policies/requirements?

So, my basic answer here is that you're not really looking at the right level of the system. Your assumption here is that file hasn't been "read" but I don't see any evidence of that. As far as the system is concerned, the file IS open and it sounds like preview opened a window and simply showed empty contents. The simplest explanation for that is that open AND read both "worked", you simply didn't return contents in "read". Or, alternatively, some other called the system returned to your driver convinced it that it shouldn't read "yet".

The place you need to look at here is what your VFS driver actually did. What VNOP's did you receive after open? Did you get VNOP_READ? If so, what did you return? What "else" happened, particularly things like ioctl calls?

Also, clarifying his point here:

Is there a specific VFS attribute that meets the system's expectation of a filesystem in terms of sandbox(7) security policies/requirements?

At this point, you're already past the sandbox system. The goal of the sandbox is to prevent unauthorized access to files, which means blocking "open". Once open succeeds, it's to late to "protect" the file.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

vnop_read does get called for Preview.app for my kext:

% sudo dtrace -f 'fbt::vnop_*_myfs/execname!="dtrace"/{}' -odtrace_myfs_preview_image_all_vnops.log

'open -a /System/Applications/Preview.app/Contents/MacOS/Preview /tmp/myfs/Through_the_pines.jpg'

% sed -n '/vnop_open/,${s!^[^a-z]*!!;s![^a-z]*$!!p;}' dtrace_myfs_preview_image_all_vnops.log
vnop_open_myfs:entry
vnop_open_myfs:return
vnop_close_myfs:entry
vnop_close_myfs:return
vnop_inactive_myfs:entry
vnop_inactive_myfs:return
vnop_getattr_myfs:entry
vnop_getattr_myfs:return
vnop_setxattr_myfs:entry
vnop_setxattr_myfs:return
vnop_getxattr_myfs:entry
vnop_getxattr_myfs:return
vnop_lookup_myfs:entry
vnop_lookup_myfs:return
vnop_open_myfs:entry
vnop_open_myfs:return
vnop_close_myfs:entry
vnop_close_myfs:return
vnop_inactive_myfs:entry
vnop_inactive_myfs:return
vnop_open_myfs:entry
vnop_open_myfs:return
vnop_read_myfs:entry
vnop_read_myfs:return
vnop_mmap_myfs:entry
vnop_mmap_myfs:return
vnop_close_myfs:entry
vnop_close_myfs:return

vnop_read returns success:

% sudo dtrace -n 'fbt::vnop_read_myfs:entry
/execname!="dtrace"/{
	self->vnop_read_arg0 = arg0;

	printf("proc: %s name: %s",
		execname,
		stringof(((struct vnop_read_args *)arg0)->a_vp->v_name)
	);
}
fbt::vnop_read_myfs:return
/execname!="dtrace" && self->vnop_read_arg0/{
	printf("proc: %s name: %s retval: %d",
		execname,
		stringof(((struct vnop_read_args *)self->vnop_read_arg0)->a_vp->v_name),
		arg1
	);
}' -odtrace_myfs_preview_image_vnop_read.log

'open -a /System/Applications/Preview.app/Contents/MacOS/Preview /tmp/myfs/Through_the_pines.jpg'

% cat dtrace_myfs_preview_image_vnop_read.log
CPU     ID                    FUNCTION:NAME
  4 146583           vnop_read_myfs:entry proc: Preview name: Through_the_pines.jpg
  2 146584          vnop_read_myfs:return proc: Preview name: Through_the_pines.jpg retval: 0

The Preview.app process' state is sleeping:

% ps acx | grep -i preview
  454   ??  S      0:00.30 Preview

The image is not displayed.

Incidentally, Google Chrome does succeed in displaying the contents of the image file. The vnops being called seem very similar with vnop_read being called multiple times.

% sudo dtrace -f 'fbt::vnop_*_myfs/execname!="dtrace"/{}' -odtrace_myfs_gc_image_all_vnops.log

'open -a /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome /tmp/myfs/Through_the_pines.jpg'

% sed -n '/vnop_open/,${s!^[^a-z]*!!;s![^a-z]*$!!p;}' dtrace_myfs_gc_image_all_vnops.log
vnop_open_myfs:entry
vnop_open_myfs:return
vnop_close_myfs:entry
vnop_close_myfs:return
vnop_inactive_myfs:entry
vnop_inactive_myfs:return
vnop_getattr_myfs:entry
vnop_getattr_myfs:return
vnop_setxattr_myfs:entry
vnop_setxattr_myfs:return
vnop_lookup_myfs:entry
vnop_lookup_myfs:return
vnop_getxattr_myfs:entry
vnop_getxattr_myfs:return
vnop_open_myfs:entry
vnop_open_myfs:return
vnop_read_myfs:entry
vnop_read_myfs:return
vnop_read_myfs:entry
vnop_read_myfs:return
vnop_read_myfs:entry
vnop_read_myfs:return
vnop_read_myfs:entry
vnop_read_myfs:return
vnop_close_myfs:entry
vnop_close_myfs:return
vnop_inactive_myfs:entry
vnop_inactive_myfs:return
vnop_pathconf_myfs:entry
vnop_pathconf_myfs:return
vnop_pathconf_myfs:entry
vnop_pathconf_myfs:return

And returning success at all times.

% sudo dtrace -n 'fbt::vnop_read_myfs:entry
/execname!="dtrace"/{
	self->vnop_read_arg0 = arg0;

	printf("proc: %s name: %s",
		execname,
		stringof(((struct vnop_read_args *)arg0)->a_vp->v_name)
	);
}
fbt::vnop_read_myfs:return
/execname!="dtrace" && self->vnop_read_arg0/{
	printf("proc: %s name: %s retval: %d",
		execname,
		stringof(((struct vnop_read_args *)self->vnop_read_arg0)->a_vp->v_name),
		arg1
	);
}' -odtrace_myfs_gc_image_vnop_read.log

'open -a /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome /tmp/myfs/Through_the_pines.jpg'

% cat dtrace_myfs_gc_image_vnop_read.log
CPU     ID                    FUNCTION:NAME
  5 144179           vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg
  4 144180          vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0
  6 144179           vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg
  1 144180          vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0
  1 144179           vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg
  0 144180          vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0
  0 144179           vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg
  3 144180          vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0

vnop_ioctl is never called for Preview.app or Google Chrome.

% grep -c vnop_ioctl dtrace_myfs_*_image_all_vnops.log
dtrace_myfs_gc_image_all_vnops.log:0
dtrace_myfs_preview_image_all_vnops.log:0

If you have any other piece of advice for me, it would be greatly appreciated.

How are you handling "mmap"? "vnop_mmap_myfs" is the main difference I see in the two logs. That would also explain why you don't see the same read series that you see in Chrome- Preview was planning to process the file through mmap instead of individual read calls.

Note that the system uses mmap quite widely, even in cases what it wasn't necessarily "obvious" that mapped I/O would occur.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks a lot for the tip about mmap.

With vnop_strategy, vnop_blockmap, vnop_offtoblk, vnop_blktooff, vnop_pagein and vnop_pageout added in, Preview.app is able to read file contents.

The idea for the algorithm for computing blockmap, offtoblk and blktooff was borrowed from Apple's open-source SMBClient implementation.

However, file memory mapping isn't working as expected.

Image files are being displayed properly in part with the rest of the image being garbled.

For pdf files, Preview.app displays a message about the file being damaged.

I suspect it's probably one or all of blockmap, offtoblk and blktooff that's at fault.

If you have any other clues for me, I'd really appreciate that.

For the sake of completeness, with the file reference counter incremented in vnop_mmap, and decremented in vnop_mnomap, image and pdf files are now being properly displayed by Preview.app.

Thanks very much Kevin Elliott for all your help.