OSX 10.11.2 breaks executing an NSTask from a sandboxed XPC service as well as more generally executables outside app bundle

I am one of the authors of Manuscripts (http://manuscriptsapp.com), a sandboxed scientific writing tool. The app supports exporting the document written with the app to LaTeX source and in case the user has a LaTeX typesetting environment installed on his system, to a PDF compiled with the pdflatex command include in LaTeX. We also take heavy use of a copy of the pandoc document transformation tool, embedded in the app bundle as an auxiliary executable, as well as some other external commandline tools used for bibliography data. Bottom line is, we need NSTask, and lots of it.


OSX 10.11.2 seems to now have broken two critical things for us with using NSTask in a sandboxed app (both of these absolutely critical scenarios worked without issue until this OS release):


1. Executing an NSTask from an XPC service corresponding to an executable that is outside the app's sandbox, URL of which the containing app has requested access to with a file open dialog and successfully requested security scoped access to. For instance: /usr/local/texlive/2015/bin/x86_64-darwin/pdflatex


The flow for doing this seems not to have been documented at all anywhere in the documentation made available by Apple. We found a solution initially via the old developer forum by an Apple user with nick "Aarie": https://devforums.apple.com/thread/142513 .


To quote solution suggested by Aarie:

"Your main app can use an app-scope bookmark to retain access to that user-chosen directory across relaunches, which I'm guessing you're already doing. Now, to get your XPC service to be able to access the directory, your main app can make a normal bookmark to that directory — not security scoped — and send it over to the XPC service. When the service resolves it, it will have access to the directory and all the files underneath it."


As of OSX 10.11.2 this no longer appears to work, instead yielding an error message "launch path not accessible" error:


<MPPDFCompiler.m:230> {com.manuscripts.compile.pdf} 2015-12-14 10:55:51.140 [ERROR] Failed to launch /usr/local/texlive/2014/bin/x86_64-darwin/pdflatex -halt-on-error -interaction nonstopmode /Users/markus/Library/Group Containers/SJK358RN87.com.manuscripts.Manuscripts/Library/Caches/33431E4C/MPPDFCompiler/manuscript.tex: launch path not accessible <MPPDFCompiler.m:292> {com.manuscripts.compile.pdf}

You can reproduce this issue with the following steps:

1. Download Manuscripts 1.0.29 or earlier:

https://rink.hockeyapp.net/api/2/apps/280aaebd99e1f12edc41b48d909db0c4/app_versions/132?format=zip&avtoken=58694a72016999f502d0b82e45893050fe4d3a8d

2. Create a new manuscript.

3. File > Export… > PDF via LaTeX.

4. Pick your LaTeX installation path root (e.g. /usr/local/texlive) with the file open dialog that pops up.

5. Observe the above log message when the export is attempted.


To see the same working, use the same app with OSX 10.11.1 or earlier.


Q: Could you please document how to approach this scenario of executing as a child process an executable that is outside application's sandbox, with NSTask in a sandboxed XPC service?


It appears also that there's even no XPC service dependency and that OSX 10.11.2 in fact more categorically can break executing binaries outside the sandbox. See https://twitter.com/bswest/status/676417150953787392



2. Executing an NSTask from an XPC service which symbolically links to another executable inside the app bundle also appears broken. An example log output from Manuscripts 1.0.29 (downloadable via https://rink.hockeyapp.net/api/2/apps/280aaebd99e1f12edc41b48d909db0c4/app_versions/132?format=zip&avtoken=58694a72016999f502d0b82e45893050fe4d3a8d):


2015-12-14 00:39:21.844 [DEBUG] Running '/Users/markus/Library/Developer/Xcode/DerivedData/Manuscripts-diwksmklnodlseggknqtdiixlpls/Build/Intermediates/UninstalledProducts/macosx/Manuscripts.app/Contents/MacOS/pandoc --from=html --to=markdown+pipe_tables-simple_tables-multiline_tables-grid_tables --email-obfuscation=none --standalone +RTS -K128m -RTS --atx-headers --no-wrap /Users/markus/Library/Containers/com.manuscripts.PressRoom/Data/Library/Caches/com.manuscripts.PressRoom/document-transformation/0421E039' <MPPandocService.m:515> {com.apple.main-thread}

2015-12-14 00:39:21.845 com.manuscripts.PressRoom[4441:48285] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'launch path not accessible'


To reproduce this issue:

1. Download Manuscripts 1.0.29 or earlier:

https://rink.hockeyapp.net/api/2/apps/280aaebd99e1f12edc41b48d909db0c4/app_versions/132?format=zip&avtoken=58694a72016999f502d0b82e45893050fe4d3a8d

2. Create a new manuscript.

3. Write some text in the manuscript and select it.

4. File > Copy to observe the log message above (copying results in pandoc being executed to complete a transformation of the document).

The pandoc executable inside the Manuscripts app bundle is inside its embedded helper commandline app bundle manuel.app: Manuscripts.app/Contents/Applications/manuel.app/Contents/MacOS/pandoc … as well as symlinked from there to the main applications' Manuscripts.app/Contents/MacOS/pandoc (this is the path from which the XPC service pressroom executes it).

Q: Could you please document how to approach this scenario of executing as a child process an executable that is outside application's sandbox, with NSTask in a sandboxed XPC service?

As a more general point I wold like to register my disbelief, and quite frankly anger, that a change like this happens in an OSX point release with no documentation to be found anywhere that the change is there or how we should actually handle as third party developers cases like pointed out above. This is not something unusual: we cannot execute LaTeX commands or pandoc in any other form but as external process, due to reasons including but not limiting licensing and an obviously massive effort being involved in shoehorning either into running inside the app's main process.


Doing so would not only be infeasible but would also break the spirit fof the security guidelines which suggest separating concerns to XPC services. For instane in our case we do not allow the component that does document transformations with pandoc or pdflatex to see resources outside the group container shared by the XPC service and the main app. As a thank you, we get issues like this that in an minor OS update cycle suddenly render the app unusable! At least we now have the option of removing the sandboxing from the XPC process as we have not made the application available in the Mac App Store yet frankly due to being terrified of events like this where the OSX introduces new (to the best of my knowledge) undocumented changes or bugs which affect the already rather scarcely documented requirements for corner cases that are actually critical for our app.

Replies

Is there a workaround for this yet or do we have to wait for 10.11.3 to come out?

To my knowledge no fix or workaround has been made since? The radar I filed about this was marked a duplicate of something else a while after I opened a developer support request regarding this and was also not given a solution that route.

The radar I filed about this was marked a duplicate of something else

Please post your bug number.

a while after I opened a developer support request regarding this and was also not given a solution that route.

What was the follow-up number for that? (If you’re not comfortable posting follow-up numbers, you can email me at the address in my signature.)

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I'm not running macOS Sierra on my development machine yet, so just a question: has this been solved in macOS version 10.12? If not, will it make its way in 10.12.1? Finally, will a new 10.11.x version be released with a fix for this (knowing Apple's history, probably not)? Or should we accept that using NSTask in sandboxed app/XPC service to run executables outside app bundle will never again be allowed?

Without a bug number it’s hard for me to check on the status of this issue. If it’s critical to you, I suggest you test it for yourself. If you don’t have a machine running 10.12 handy, you can always install it in a VM.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

The lack of testing comes from my laziness, as it isn’t an urgent issue for me (yet). Anyway, I have tested it and I can confirm the bahaviour in 10.12 is the same as in 10.11.2+: a binary outside of sandboxed app bundle cannot be run using NSTask, even though the url/path to the binary is acquired using regular sandboxing mechanisms (entitlements/ Powerbox). I think this should be fixed or if it is “a feature” (the famous answer in the bug report “this works as intended”) then it should be clearly documented at least.


NSTasking a 3rd party binary is essential for the application I’m working on, because it does some processing, the result of which should be presented to the user using the app/binary of his/her choice (chosen using Powerbox). At the moment, my workaround is a dedicated non-sandboxed XPC service, whose sole purpose is to run binaries using NSTask. But I’d like to avoid this approach if possible, for numerous reasons:


1) I’d like my application to be completely sandboxed

2) I’d like to keep an option to put the application in the Mac Apple Store (I haven’t decided yet whether to do it or not, but if this issue is not resolved I suppose the decision has been made for me already)

3) I’d like to ditch that additional XPC service and use NSTask from already existing sandboxed XPC service, which does the processing I’ve mentioned above


I haven’t reported any bug on this (was done by another person in this thread, mz2), but if necessary I’ll do it. It will probably be marked as a duplicate of something else, but then I’ll have bug numbers to report back. Just one question about reporting this, to make things faster; do I need to provide a sample application reproducing the issue, or just referring to this forum discussion would be enough?


Thanks for your interest in this.

--Dragan

Thanks for the detailed post; most helpful.

[a bug was filed] by another person in this thread, mz2

Right, but they didn’t post the bug number )-:

Just one question about reporting this, to make things faster; do I need to provide a sample application reproducing the issue, or just referring to this forum discussion would be enough?

You probably don’t need to provide a sample app but, given that such an app would be pretty simple to create, I recommend you do so.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Done!! The bug number is: 28844650


I have also provided two sample projects for the same application, one using non-sandboxed XPC service to launch a tool using NSTask, the other one using sandboxed XPC service (same code, same everything, just sandboxed), which fails with the "Exception: launch path not accessible", so it should be easy to reproduce. Other detailed explanations of what the application does and results can be found in there.


I really appreciate your interest and help with this. And I sincerely hope this will be modified/fixed not only in Sierra, but in one last update of El Capitan (but I already know the latter is probably just a pipe dream).

OK, for you eskimo, and for anyone still interested in this issue. This is the response I've received through Apple Bug Reporter:


"We have NSOpenPanel and NSSavePanel for reading or writing files at paths that the user selects, respectively. There is no panel for choosing a path to execute nor are there any plans to add one. It would be misleading if AppSandbox allowed the app to execute a file at a path that the user selected from “opening” and, as such, it has not been allowed since AppSandbox first shipped in 10.7.

Thank you for your feedback, it is noted. Engineering has determined that there are currently no plans to address this issue. We are now closing this bug report. If you have questions or comments about the resolution, please update your bug report with that information so we can respond."

So, there goes my eventual plan to put my app in the MAS :-( The only thing I don't understand is "it has not been allowed since AppSandbox first shipped in 10.7", but it used to work in versions prior 10.11.2, as confirmed by more people here. Probably THAT was what Apple considered being a bug.

The

com.apple.security.temporary-exception.files.absolute-path.read-only
entitlement (and its home relative and read write variants) allows exec'ing a process under the corresponding directory for the entitlement. I found that with that entitlement, I can use NSTask within a App Sandboxed XPC service to run binaries that are outside the app sandbox.