XPC Services cannot open/read files in Catalina

More test Info:

Sandboxed host app + Sandboxed XPC ==> XPC service has no ability to open files

Non-Sandboxed host app + Sandboxed XPC ==> XPC service has no ability to open files

Non-Sandboxed host app + Non-Sandboxed XPC ==> XPC service can open files ==== YEAH !!! ======

Sandboxed host app + Non-Sandboxed XPC ==> This configuration is no longer allowed on Mac App Store

Sandboxed XPC services do NOT have their own sandbox container. Apparently, they are expected to share the sandbox container of the

host app that invokes them.

So the Catalina bug is that XPC services can no longer inheirit the sandbox container from their host app.

So Sandboxed XPC services have no access at all to the file system in current versions of Catalina.

If you remove the sandbox entitlement, then file system access is available.

In both instances, the XPC service did not "inherit" the sandbox state of its host app. This violates the

com.apple.security.inherit

entitlement.

com.apple.security.inherit

If your app employs a child process created with either the

posix_spawn

function or the

NSTask

class, you can configure the child process to inherit the sandbox of its parent.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

So that means that an XPC service cannot open the file the host app intentionally sends it. The XPC service is packaged inside the host app's bundle. The host app and its bundled XPC service were designed to work together.

If you look inside ~/Libary/Containers, no container for the XPC service is ever created. XPC services apparently do not have a sandbox container directory.

The XPC service is typcially not a app itself, it has no user interface. It does not use AppKit. It certainly does not have the ability to invoke a NSSavePanel, to get access to an arbitrary file.

Apparently, in previous versions of Mac OS, Mojave and High Sierra, sandboxed XPC services COULD open files that were inside the host app's container.

Apple's CHANGE-OF-DESIGN for Catalina is most unfortunate. It now makes XPC services basically useless for most applications.

Why is this important ? ================================

My own app took advantage of this file sharing capability heavily.

My app delegates video rendering to an XPC process and literally create thousands of image files, one each per video frame.

One minute of video has 1800 video frames. When you add motion blur, you are literally creating 50,000 distinct images in a single minute of video. An average song lasts 4 1/2 minutes, so to create fluid video animations for the average song, takes approx 1/4 million images.

A small memory leak per rendered image mutliplied by 250,000 becomes a very, very big memory leak. By putting the problematic render engine inside a separate XPC service, that isolates the big memory leak to the XPC service process. The app tells the XPC service to render 200 video frames. After the 200 video frames are rendered, the XPC service is shutdown and restarted.

This is far better than having the app consume all of the free memory on the Mac, the only alternative now available on Catalina.

So, Eskimo, are you clearly stating this:

Sandboxed XPC services are not allowed to access the file system. Opening files by a sandboxed XPC service is explicitly prohibited by the system.

If you are not stating this, what is the workaround?

How is it possible for an sandboxed XPC service to open one or more files, or directories?

How is it possible for an sandboxed XPC service create new files inside a directory that the host app designates?

My users have been used to this XPC rendering architecture for the last 2 major versions of Mac OS with my app.

Do I have to tell them: sorry you upgraded to Catalina, but you are stuck.

You get to babysit the render process and restart the app when the memory leak gets too big.

I can't emphasize enough how angry my users are about this.

I have checked the Entitlements Reference.

Apparently the last revision dates back to 2017-03-27. So no design change for Catalina in all of this.

The "App Sandbox in Depth" article referenced by the "Entitlements Reference": Its last revision is 2016-09-13

But XPC service file access works perfectly in High Sierra and Mojave, but not in Catalina.

With zero published design change, this XPC service's inability to open a URL designated by the host app in Catalina

can only be construed as a serious, devastating bug.

The original implementation worked correctly. Catalina's implementation does not.

One more thing:

XPC services are a great. In the past, they worked extremely well.

They were very easy to work with in Xcode. The ability to debug both sides, the host app and the XPC service, at the same time in Xcode is marvelous.

In production, with my app, on High Sierra and Mojave, using the XPC service to isolate a big memory leak worked so extremely well.

Please don't nerf sandboxed XPC services. It is one of the best examples of quality engineering in Apple's arsenal.

If there has been architetural design change for XPC, please make sure that they have some way to share files/folders with their host apps.

OK, that’s a lot of posts with a lot of questions. Rather than address them piecemeal, I’m going to focus on your requirements. You wrote:

My app delegates video rendering to an XPC process and literally create thousands of image files, one each per video frame.

So, you have a design where the XPC Service creates files and you want the app to consume them?

If so, the easiest option here is to set up a shared app group between your app and your XPC Service. That gives you a container (you can find its location using

-containerURLForSecurityApplicationGroupIdentifier:

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thank you. I will give that a try.

Shared app groups helped me to avoid a heart attack when running into this problem. Wish this would be more clear documented. Since editorial explaining PDF documentation was removed everything totally sucks here.