When dtrace(1)-ing archive extraction over smbfs I overlooked a vnop_pathconf call, which turned out to be the missing link. 0 -> smbfs_vnop_pathconf AUHelperService-1676 -> extractme a_name:10 ;_PC_NAME_CHARS_MAX 0 <- smbfs_vnop_pathconf AUHelperService-1676 -> extractme a_name:10 -> 255 0 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 nameiop:0 0 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 -> 2 ;ENOENT 0 -> smbfs_vnop_lookup AUHelperService-1676 -> TheRooT/extractme/misc 2 nameiop:0 0 <- smbfs_vnop_lookup AUHelperService-1676 -> TheRooT/extractme/misc 2 -> 0 0 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 nameiop:0 0 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 -> 2 0 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 nameiop:0 0 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 -> 2 0 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme.sb-27bcb433-Yj4plp/misc nameiop:0x2 ;DELETE 0 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme.sb-27bcb433-Yj4plp/misc -> 0 0 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 nameiop:0x3 ;RENAME 1 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 -> -2 ;EJUSTRETURN 1 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme.sb-27bcb433-Yj4plp/misc nameiop:0x2 1 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme.sb-27bcb433-Yj4plp/misc -> 0 1 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 nameiop:0x3 2 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 -> -2 2 -> smbfs_vnop_rename AUHelperService-1676 -> extractme.sb-27bcb433-Yj4plp/misc -> extractme/nil 2 <- smbfs_vnop_rename AUHelperService-1676 -> extractme.sb-27bcb433-Yj4plp/misc -> extractme/nil -> 0 1 -> smbfs_vnop_lookup AUHelperService-1676 -> TheRooT/extractme/misc 2 nameiop:0 2 <- smbfs_vnop_lookup AUHelperService-1676 -> TheRooT/extractme/misc 2 -> 0 2 -> smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 nameiop:0 3 <- smbfs_vnop_lookup AUHelperService-1676 -> extractme/misc 2 -> 0 With vnop_pathconf implemented archive extraction over the same directory multiple times on my custom filesystem works as expected. Thanks.

Thanks for your input. No, my filesystem is not based on FUSE. It's a port of a distributed filesystem from a UNIX-like OS to macOS. The original implementation mostly uses locking to protect the RPCs involved in each file operation. On macOS, this doesn't seem sufficient. Thus my query. The Linux kernel documentation for VFS describes the locking rules that must be followed by all filesystems. FreeBSD filesystems extensively employ locking in their VFS implementations. So I assumed that macOS must have its own locking rules for VFS as well. It would be helpful to understand what they are, so my filesystem can adhere to them properly.

Turns out, it was my lookup algorithm that was to blame. Once adjusted as described here rm(1) was no longer complaining and running to completion. Thanks for your help.

Thanks a lot for the tip Kevin. Basic testing does work fine. terminal1: % echo line1 >file.txt terminal1: % ed file.txt ;open file.txt in ed terminal2: % rm file.txt terminal1: % ed file.txt ;close file.txt in ed dtrace output for terminal tests Finder 1. Create file.rtf using TextEdit. 2. Close file.rtf using TextEdit. 3. Open file.rtf using TextEdit. 4. Remove file.rtf via Finder. 5. Close file.rtf using TextEdit. dtrace output for Finder tests Onto a debugging session of rm(1) as you suggested.

Thanks for replying. I had a closer look at the signature of select and realised that there was no way I knew of that I could get a file descriptor to pass into it within the kernel.

Thanks very much for clarifying things, Kevin. Coincidentally, I too was thinking of writing my own tests for the VNOPs implemented. Alexander Kapshuk

Thanks for a prompt response. I disabled local locking and implemented a vnop_advlock of my own. AJAsystemtest issues a POSIX advlock request with vnop_advlock_args.a_flags being set to F_POSIX. What I don't understand is why I'm only getting F_UNLCKs, but no F_SETLKs. A little help clarifying why this is, as well as being pointed in the right direction would be much appreciated.

Passing the smaller value of buf_count and the difference between the file size and the current offset into the file, instead of passing the value returned by buf_count, into myfs_write function fixes the problem. Thanks very much Kevin Elliott for the tip.

For the sake of completeness, with the file reference counter incremented in vnop_mmap, and decremented in vnop_mnomap, image and pdf files are now being properly displayed by Preview.app. Thanks very much Kevin Elliott for all your help.

Thanks a lot for the tip about mmap. With vnop_strategy, vnop_blockmap, vnop_offtoblk, vnop_blktooff, vnop_pagein and vnop_pageout added in, Preview.app is able to read file contents. The idea for the algorithm for computing blockmap, offtoblk and blktooff was borrowed from Apple's open-source SMBClient implementation. However, file memory mapping isn't working as expected. Image files are being displayed properly in part with the rest of the image being garbled. For pdf files, Preview.app displays a message about the file being damaged. I suspect it's probably one or all of blockmap, offtoblk and blktooff that's at fault. If you have any other clues for me, I'd really appreciate that.

vnop_read does get called for Preview.app for my kext: % sudo dtrace -f 'fbt::vnop_*_myfs/execname!="dtrace"/{}' -odtrace_myfs_preview_image_all_vnops.log 'open -a /System/Applications/Preview.app/Contents/MacOS/Preview /tmp/myfs/Through_the_pines.jpg' % sed -n '/vnop_open/,${s!^[^a-z]*!!;s![^a-z]*$!!p;}' dtrace_myfs_preview_image_all_vnops.log vnop_open_myfs:entry vnop_open_myfs:return vnop_close_myfs:entry vnop_close_myfs:return vnop_inactive_myfs:entry vnop_inactive_myfs:return vnop_getattr_myfs:entry vnop_getattr_myfs:return vnop_setxattr_myfs:entry vnop_setxattr_myfs:return vnop_getxattr_myfs:entry vnop_getxattr_myfs:return vnop_lookup_myfs:entry vnop_lookup_myfs:return vnop_open_myfs:entry vnop_open_myfs:return vnop_close_myfs:entry vnop_close_myfs:return vnop_inactive_myfs:entry vnop_inactive_myfs:return vnop_open_myfs:entry vnop_open_myfs:return vnop_read_myfs:entry vnop_read_myfs:return vnop_mmap_myfs:entry vnop_mmap_myfs:return vnop_close_myfs:entry vnop_close_myfs:return vnop_read returns success: % sudo dtrace -n 'fbt::vnop_read_myfs:entry /execname!="dtrace"/{ self->vnop_read_arg0 = arg0; printf("proc: %s name: %s", execname, stringof(((struct vnop_read_args *)arg0)->a_vp->v_name) ); } fbt::vnop_read_myfs:return /execname!="dtrace" && self->vnop_read_arg0/{ printf("proc: %s name: %s retval: %d", execname, stringof(((struct vnop_read_args *)self->vnop_read_arg0)->a_vp->v_name), arg1 ); }' -odtrace_myfs_preview_image_vnop_read.log 'open -a /System/Applications/Preview.app/Contents/MacOS/Preview /tmp/myfs/Through_the_pines.jpg' % cat dtrace_myfs_preview_image_vnop_read.log CPU ID FUNCTION:NAME 4 146583 vnop_read_myfs:entry proc: Preview name: Through_the_pines.jpg 2 146584 vnop_read_myfs:return proc: Preview name: Through_the_pines.jpg retval: 0 The Preview.app process' state is sleeping: % ps acx | grep -i preview 454 ?? S 0:00.30 Preview The image is not displayed. Incidentally, Google Chrome does succeed in displaying the contents of the image file. The vnops being called seem very similar with vnop_read being called multiple times. % sudo dtrace -f 'fbt::vnop_*_myfs/execname!="dtrace"/{}' -odtrace_myfs_gc_image_all_vnops.log 'open -a /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome /tmp/myfs/Through_the_pines.jpg' % sed -n '/vnop_open/,${s!^[^a-z]*!!;s![^a-z]*$!!p;}' dtrace_myfs_gc_image_all_vnops.log vnop_open_myfs:entry vnop_open_myfs:return vnop_close_myfs:entry vnop_close_myfs:return vnop_inactive_myfs:entry vnop_inactive_myfs:return vnop_getattr_myfs:entry vnop_getattr_myfs:return vnop_setxattr_myfs:entry vnop_setxattr_myfs:return vnop_lookup_myfs:entry vnop_lookup_myfs:return vnop_getxattr_myfs:entry vnop_getxattr_myfs:return vnop_open_myfs:entry vnop_open_myfs:return vnop_read_myfs:entry vnop_read_myfs:return vnop_read_myfs:entry vnop_read_myfs:return vnop_read_myfs:entry vnop_read_myfs:return vnop_read_myfs:entry vnop_read_myfs:return vnop_close_myfs:entry vnop_close_myfs:return vnop_inactive_myfs:entry vnop_inactive_myfs:return vnop_pathconf_myfs:entry vnop_pathconf_myfs:return vnop_pathconf_myfs:entry vnop_pathconf_myfs:return And returning success at all times. % sudo dtrace -n 'fbt::vnop_read_myfs:entry /execname!="dtrace"/{ self->vnop_read_arg0 = arg0; printf("proc: %s name: %s", execname, stringof(((struct vnop_read_args *)arg0)->a_vp->v_name) ); } fbt::vnop_read_myfs:return /execname!="dtrace" && self->vnop_read_arg0/{ printf("proc: %s name: %s retval: %d", execname, stringof(((struct vnop_read_args *)self->vnop_read_arg0)->a_vp->v_name), arg1 ); }' -odtrace_myfs_gc_image_vnop_read.log 'open -a /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome /tmp/myfs/Through_the_pines.jpg' % cat dtrace_myfs_gc_image_vnop_read.log CPU ID FUNCTION:NAME 5 144179 vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg 4 144180 vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0 6 144179 vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg 1 144180 vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0 1 144179 vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg 0 144180 vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0 0 144179 vnop_read_myfs:entry proc: Google Chrome name: Through_the_pines.jpg 3 144180 vnop_read_myfs:return proc: Google Chrome name: Through_the_pines.jpg retval: 0 vnop_ioctl is never called for Preview.app or Google Chrome. % grep -c vnop_ioctl dtrace_myfs_*_image_all_vnops.log dtrace_myfs_gc_image_all_vnops.log:0 dtrace_myfs_preview_image_all_vnops.log:0 If you have any other piece of advice for me, it would be greatly appreciated.

Thanks very much for clarifying the proper use of open(1) and for being willing to help. Now, when launching Preview.app through 'open(1) -a' or by double clicking the image file in Finder, Preview.app launches: % open -a /System/Applications/Preview.app/Contents/MacOS/Preview /tmp/myfs/Through_the_pines.jpg lsof(8) reports the image file as being opened by Preview.app: % lsof -p `pgrep -xi preview` | grep Through Preview 2825 developer 3r REG 54,2 2396454 3001938654 /private/tmp/myfs/Through_the_pines.jpg But the contents of the image file aren't being displayed. System logs have the following records: 2024-06-20 09:38:25.821809+0300 0x548e Default 0x0 2823 2 open: (LaunchServices) [com.apple.launchservices:open] Opening document <FSNode 0x600003f790a0> { isDir = n, path = '/private/tmp/myfs/Through_the_pines.jpg' } with application <FSNode 0x600003f79160> { isDir = y, path = '/System/Applications/Preview.app' } 2024-06-20 09:38:26.094972+0300 0x54da Default 0x0 2825 0 Preview: (Foundation) [com.apple.foundation.filecoordination:claims] Read options: 0 -- URL: file:///private/tmp/myfs/Through_the_pines.jpg -- purposeID: 78FD7C2A-102A-4679-A0FE-4854596E124C -- claimID: 2D45E1F1-8533-4F26-ADE7-22E92590C39A The same behaviour is observed with pdf files. As my filesystem is a network one, I thought I'd go and see if https://github.com/apple-oss-distributions/SMBClient offers anythings useful, as opening image and/or pdf files works there. In the meantime, if you have any more tips for me, it would be much appreciated. Is there a specific VFS attribute that meets the system's expectation of a filesystem in terms of sandbox(7) security policies/requirements?

When launching Preview.app on the command line, I get an 'Operation not permitted' error returned. % /System/Applications/Preview.app/Contents/MacOS/Preview /Volumes/myfs/Through_the_pines.jpg 2024-06-17 14:23:16.826 Preview[2745:37915] PVImageContainer initWithURL:file:///Volumes/myfs/Through_the_pines.jpg failed, error = Error Domain=NSCocoaErrorDomain Code=257 "The file “Through_the_pines.jpg” couldn’t be opened because you don’t have permission to view it." UserInfo={NSFilePath=/Volumes/myfs/Through_the_pines.jpg, NSUnderlyingError=0x6000022302a0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}} File permissions allow reading for all users: % ls -l /Volumes/myfs/Through_the_pines.jpg -rw-rw-r--@ 1 developer staff 2396454 Mar 25 10:48 /Volumes/myfs/Through_the_pines.jpg Am I right in assuming it's sandbox related?

Thanks for replying. I've run stat(1) asynchronously to verify that two processes querying the same file residing on my filesystem get the identical inode value. See below. % stat -f 'inode: %i' t.txt&; stat -f 'inode: %i' t.txt& [1] 1978 [2] 1979 inode: 1954135111 [2] + done stat -f 'inode: %i' t.txt inode: 1954135111 [1] + done stat -f 'inode: %i' t.txt What are the system expectations of a filesystem that my implementation should meet? Is it some struct vfs_attr value that should be set? It has to be something specific, shouldn't it? Is there anything else you could recommend in terms of troubleshooting this? Thanks.