The part that I don't understand is why an attribute designed to warn users of an untrusted application would have the side effect of preventing the application from being deleted (or at least making it more difficult). Granted that users would not delete the application but instead move it to the trash, but from a file sharing client apparently I no longer have that option. Is it the case that I am able to delete the application on the server because Terminal has full disk access, but I cannot delete it remotely because a file sharing client does not have similar permission?

Using Howard Oakley's xattred program, I examined the original application on the client. (I built this application on the client machine.) The program reports that every file and directory in the application has the com.apple.provenance attribute with identical values. More interesting is that the file Contents/runtime-x86/Contents/_CodeSignature/CodeResources has the com.apple.quarantine attribute, with the value 0083;673245b6;Safari;71F7C727-7831-4DBB-B9FA-13D14B18F09D. I'm guessing that is because I copied this directory from a downloaded JDK. The corresponding file under Contents/runtime-arm does not have this attribute. Presumably, the process by which I obtained this JDK runtime was different. Is this the cause of the problem? Should I delete the com.apple.quarantine attribute? What process should I use to extract a runtime from a JDK? It seems that files in a JDK routinely have this attribute. A JDK is not an application, however, so it never gets cleared?

The application is copied using ditto.

I tried to replicate the situation on the server, but the newly copied application files do not have xattrs, so I am unable to say what error might have been reported. The file that I described had a value of 01 02 00 F5 28 1A 84 40 15 BA C9 when I inspected it on the server. However, its current value is empty. Other files in that application had the same attribute and it seems they all now have empty values. I do not see any other attributes.

I'm not sure there is a real difference. The client may have gotten an error when it tried to read the extended attributes of runtime-arm/Contents. In any case, the extended attribute is com.apple.provenance.

I am somewhat surprised that moving the application to a directory that is not and has never been displayed by Finder (before trying to delete the application) does not fix the problem.

I can confirm that after rebooting the client system, I get the same error report from Terminal on the client system. Mac-mini:testSystem alan$ ll -R VAquaManager.app.1734032083499 total 32 drwxr-xr-x@ 1 alan staff 16384 Dec 12 11:34 Contents VAquaManager.app.1734032083499/Contents: total 64 drwxr-xr-x@ 1 alan staff 16384 Dec 8 09:37 runtime-arm drwxr-xr-x@ 1 alan staff 16384 Dec 8 09:37 runtime-x86 VAquaManager.app.1734032083499/Contents/runtime-arm: total 32 drwxr-xr-x 1 alan staff 16384 Dec 12 11:34 Contents VAquaManager.app.1734032083499/Contents/runtime-arm/Contents: total 0 ls: fts_read: Permission denied Using Terminal on the server system, I get: alan@Alans-iMac testSystem % ll -R VAquaManager.app.1734032083499 total 0 drwxr-xr-x@ 4 alan staff 136 Dec 12 11:34 Contents VAquaManager.app.1734032083499/Contents: total 0 drwxr-xr-x@ 3 alan staff 102 Dec 8 09:37 runtime-arm drwxr-xr-x@ 3 alan staff 102 Dec 8 09:37 runtime-x86 VAquaManager.app.1734032083499/Contents/runtime-arm: total 0 drwxr-xr-x@ 2 alan staff 68 Dec 12 11:34 Contents VAquaManager.app.1734032083499/Contents/runtime-arm/Contents: total 0 VAquaManager.app.1734032083499/Contents/runtime-x86: total 0 drwxr-xr-x@ 3 alan staff 102 Dec 12 11:35 Contents VAquaManager.app.1734032083499/Contents/runtime-x86/Contents: total 0 drwxr-xr-x@ 2 alan staff 68 Dec 12 11:35 Home VAquaManager.app.1734032083499/Contents/runtime-x86/Contents/Home: total 0

To clarify, I don't think APFS itself is a key factor. How long is your current retry timing? Certainly comparing local and network volumes, much longer timeouts are often required for a network file system. My timeout is currently 20 seconds, but I have previously used 60 seconds. I have never had a similar problem with local directory deletion. When local directories fail to delete, it is because of a spontaneously created .DS_Store file. What does "viewed" here actually mean? For example, using Terminal on the client system to list the directory, delete the directory, or list the parent directory. What's the file system on the SSD? APFS, HFS+, or something else? HFS+. I use this volume with various old macOS releases back to 10.10. On the delete side... what's the specific command/function your executing? When diagnosing from Terminal, rmdir. Probably the same system call from my application. As I mentioned above, I would not assume that the server is the failure point here. I believe the problem survives a client reboot. I will double check that after I submit this reply. Also, is the Finder actively open and monitoring these directories while all this is happening? The Finder doesn't normally create .DS_Store file for directories it isn't actually interacting with. Yes (on the server system). And I have observed .DS_Store files after the deletion attempt that were not there before the deletion attempt. (The Finder is open because after updating the applications on the server system, the next thing I will do is run some of them.) One more thing: I successfully implemented a workaround where the client moves the application to the (remote) trash instead of deleting it. That worked when the server system was 14.7.1, but fails after updating it to 14.7.2 (Operation not permitted). My only current workaround is to rename the application (within the same directory) and leave it there for me to delete manually using the Finder on the server system.

First off, any file system is inherently shared data structure that many process can modify at anytime. Strictly speaking, it isn't possible to create a program that is GUARANTEED to be able to delete any given directory- file deletion is generally slower than creation, so it's entirely possible for a process to create files faster than you can create them, preventing deletion. Of course, but that is not what is happening here. At most one file is being created, a .DS_Store file. Having encountered this problem before in a strictly local context, I changed my program to retry and that has worked (with local files). If the problem is APFS delaying the deletion of the directory, then perhaps I need a longer timeout on my retry loop? The real problem is not the failure to delete the directory, it is inconsistent state that is created (as viewed by a network client). Apparently, the file server believes the directory exists (it shows up when remotely listing the parent) even after the directory ceases to exist (as viewed on the server file system). At the very least, when the network client tries to list or delete this directory, the file server should notice that the directory no longer exists, update its state and return a nonexistent file error. Instead, it returns a permissions error but does not update its state, so any future attempt to list or delete the directory will get the same error. This bad state apparently lasted for days. I consider this a bug in the file server. I probably will modify my program to move the application before trying to delete it. I don't like the idea of adding macOS specific code to an application that is currently not OS specific, but that may be the only option. I believe that macOS specific code is needed to find a location (trash) where the Finder (or APFS) will not continue to try to calculate the application size. (Does moving the application to the trash stop APFS from recalculating its size?) The directories are not in the home directory. They are on an external SSD volume.

Ordinary Unix calls...

As I already mentioned, the client and server are both macOS systems and the server volume is offered using macOS file sharing. The connection is a few feet of ethernet. There are no multiple shares. As you suggest, the persistence seems to be on the server side. Not the file system, but the network server. Using Unix tools on the server to view the directories produces the expected results, except for the delay. Using Unix tools on the client to view or delete the directory fails. This failure still occurs even now. As I mentioned, the client is using ordinary Unix system class to delete files and directories. I just noticed that in some other applications that were installed by my program there is a .DS_Store file in the corresponding location. I do not have Calculate All Sizes enabled on the server system, but I notice that Finder displays the sizes of bundled applications. I suspect there is a problem with Finder trying to recalculate the size of the application concurrently with my program trying to delete the application. The problem might involve creating a .DS_Store file (a known problem) in D or it might just introduce a delay in the actual deletion of S. My program already has retry logic to handle the spontaneous creation of .DS_Store files during a directory tree deletion, but it cannot handle the permanent failure introduced by file sharing.

Kevin, I thank you for your response. I believe progress is happening. For clarity, let me call the arm machine the client system and the x86 machine the server system. I am deleting a directory tree on the server system from a Java application running on the client system. Java uses basic system calls (rmdir and unlink) to delete items. I put a breakpoint on the exception handler and discovered an interesting situation. The failure on directory deletion is directory not empty. That should not happen because before attempting to delete the directory, my program deleted its contents. When I examine the directory that I could not delete (D) in Terminal on the server system, it is indeed not empty. It contains an empty subdirectory (S), which my program previously "deleted". A few seconds later, directory S disappeared (as viewed in Terminal on the server system)! It appears that there is a race condition. The operation to delete S apparently succeeded, but did not take effect immediately. The operation to delete D somehow overtook the previous operation and failed as a result. From Terminal on the client system, S appears to exist but trying to list its contents fails with the fts_read error. I get the same error if I open a new Terminal window and navigate to D and try to list S. If I unmount the volume and reconnect, I see the same bad state in Terminal. Listing D shows S. Listing S gets the fts_read error. Is this a bug or am I doing something wrong? Is there a reliable way to work around this problem?

I can confirm that the problem I was having has been fixed in Xcode 16.1 and Command Line Tools 16.1. I was confused before because although Xcode 16.1 was installed, I was actually using Command Line Tools 16.0. You can tell if the library is correctly built if otool -l shows a hard link to the Quartz framework and no link to the QuickLookUI framework.

A very crude and heavy-handed workaround: Add a weak link to QuickLookUI as I described previously. Run the executable with DYLD_FORCE_FLAT_NAMESPACE=1. The essence of the problem is that my library compiled with recent releases of Xcode identifies QLPreviewView as belonging to the namespace assocated with QuickLookUI, but on macoS 11 and earlier, it is in the namespace associated with Quartz, so it is not found using the default two-level namespace.

Alas, I spoke too soon. Although adding the weak link to QuickLookUI allows the library to load on macOS 11, it does not link properly. The code fails trying to create a QLPreviewView.