macOS newbie building internal updater

Is this all because the DMG images are not signed (yet), so hdiutil is failing?

No. In general, macOS does not require that you sign disk images.

There are so many things that could be going wrong here. Let’s start with the first error: In step 3, when the disk image fails to mount, what error does hdiutil print?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you for the reply.

ERROR 12/10/2023 11:11:05.362 Process failed: /usr/bin/hdiutil attach /var/folders/jv/bf5sm5951ylg9wts7tvpzhvh0000gp/T/ELS_Updater/ELS-4.0.0-development-2312081735.dmg -mountroot /var/folders/jv/bf5sm5951ylg9wts7tvpzhvh0000gp/T/ELS_Updater
java.io.IOException: Cannot run program "/usr/bin/hdiutil": error=316, spawn failed

• Have not been able to find a detailed description of error 316.
• The process uses the system temp directory retrieved with Java System.getProperty("java.io.tmpdir")
• Both the Main .app and updater have an embedded OpenJDK JRE.
• The DMGs are built with DMG Canvas, https://www.araelium.com/dmgcanvas.
• Both programs are launched using a binary Java launcher from DMG Canvas.
• This is happening in the updater that was just downloaded and attached, after it downloads the update.
  • The thread waits 2.5 seconds for the download to settle.
  • Then it attempts to attach the update DMG.
  • Three attempts are made separated by 1.5 seconds
  • Also happens if the updater is executed manually.
  • The hdiutil attach command line works when executed by hand.
• It also fails to restart the Main .app with an error=2 No such file or directory.
  • That error is incorrect, the file and directory exist.
  • The open command line works when executed by hand.
• The entire updater process works when executed locally in a debugger (IntelliJ) executing the commands one line at a time.
  • That is why diagnosing this has been particularly difficult.

My only guess is there are nuances to macOS I do not know yet.

What is macOS doing in this mix, e.g. scanning the DMG? Are there security issues?

Thank you for your time and effort. This one isn't easy.

spawn failed suggest that this isn’t something wacky related to disk images but that it can’t even start the process. You wrote:

Have not been able to find a detailed description of error 316.

That doesn’t look like the kinda of Apple error code that I’d expect to see when, say, calling posix_spawn.

If you temporarily modify your code to run hdiutil with no arguments, what do you see?

That’s a useful diagnostic test because I’d expect to see one of two outcomes:

• You continue to see the error you’re seeing.

• You see hdiutil start and print its help:

% hdiutil
hdiutil: missing verb
<verb> is one of the following:
…

If you see the first outcome, you know that this has nothing to do with the disk image you’re working with.

Both the Main .app and updater have an embedded OpenJDK JRE.

That’s open source, right? Can you dig through that to find the meaning of that error? If a macOS API, like posix_spawn, is failing, I’d like to know what error it’s returning.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

If you temporarily modify your code to run hdiutil with no arguments, what do you see?

Nothing at all with hdiutil help. I'm not capturing stderr so that may be why. No arguments returns a 1 error code whereas hdiutil help returns 0.

That’s open source, right?

Correct. See: https://openjdk.org/. I choose to not play Larry Elison's (Oracle) games.

Status

Your mention of Posix sent me down a different path. I neglected to mention I'm using JDK 8. That seems to be a problem with System.getRuntime().exec() that in JDK 8 uses fork() instead of posix_spawn().

After a great of deal testing and gyration I dropped JDK 11 into the Updater build and it appeared to work.

However my build process originates in Linux. So to fully test this notion I have to upgrade Linux, Windows and Mac to JDK/JRE 11 and make some code changes related to Collection sorting.

Am in the process of doing that now. Will report findings and solution if this works.

A Note

I will have further questions. In particular now, when going through the full process of building a deployment, uploading to GitHub. Then downloading and installing the base Main .app I receive a macOS dialog about not being able to verify the author and it not allowing the application to be executed. After 2-3 attempts it adds a button for "Run anyway".

Being new to macOS is that common practice? Will I ultimately need to sign-up, pay, and get a certificate to sign this?

I'm running Big Sur and am not familiar with the latest macOS conventions or security requirements. Would like to support all active macOS systems, again given the nature of this application (trying not to expound on that).

Thank You!

Once again, thank you for your time and effort. These are the last stumbling blocks to releasing a Beta of ELS after over 2 years of work on version 4.0 adding the desktop appliication and about 50K lines of code.

(( I couldn't do your job that you are very good at, not enough patience 😁 Hazzah Mighty Quinn the Eskimo! Oddly, I was called that back in the '60s-'70s when I was a lead player in (you never heard of) Mid-West rock bands. Respects. ))

Updated everything to Java 11. Required minor code changes.

hdiutil attach of the downloaded update still fails. Patterns and error codes are different.

Will assemble a set of results and report tomorrow.

(( btw - using ELS-Navigator, obviously not my name, because I'm old and hope to pass this project to others. So have the GitHub site, etc. setup to make the transition easier ))

Have run a series of tests with this updated Java 11 setup. It seems it now fails because the original software, the updater, and the update itself are being downloaded from GitHub.

That is further supported by getting an error when it is first installed. Getting past that requires trying 2-3 times with right-click Open on the .app.

1. Works end-to-end when DMG files are installed from local, and copied from local instead of downloaded during an "update".
2. The downloaded updater cannot be mounted with exception: Cannot run program "/usr/bin/hdiutil": error=316, posix_spawn failed
3. Found this explanation: error 316 smInitStatVErr: The InitStatusV field was negative after primary or secondary init
   1. Have no idea what that means.
4. A copy of the mount command works when executed by hand.
5. When updater is mounted by hand and executed it cannot mount the update after download, same exception.
6. In both cases the updater cannot restart the .app with exception: java.io.IOException: Cannot run program "/usr/bin/open" (in directory "/Applications/ELS.app"): error=2, No such file or directory
7. A copy of the open command works when executed by hand.
8. Again, see #1.

• What is Gatekeeper's role in all of this? Seems it's sandboxing or something and not allowing system-level access to commands.
• Is there a better procedure for applying updates to an .app?
• Am I down to figuring-out how the get and use a signing certificate and whatever "notarize" is?

What is Gatekeeper's role in all of this?

Gatekeeper is the user-level term for a variety of trusted execution subsystems on macOS. You can learn more from the various links on the Trusted Execution Resources page. Make sure to follow the links to the other “Resources” pages.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Will do. Thanks for the link.

So you agree, given the behavior of the code, that macOS security requirements must be met for this to work correctly, as opposed to there being a bug?

Must admit I have difficulty with spending money for an application that is Open Source and free. Would guess that reduces the available free software for Apples.

Thanks again for your time. Will come back to this once it's working and report.

A related question -- upon further testing of the application I cannot reach it from another system to test cross-platform interoperability when run in client/server mode.

It will only use 127.0.0.1 no matter which technique is used to executed it - even in the IDE debugger. And it changes the listener port.

1. Why does macOS do that?

2. Is there a setting or something to change that behavior?

This makes testing cross-platform interoperability impossible.

One last question if you please ...

I'm using a Macbook Pro 13" Retina Late 2013 with Big Sur 11.7.10 which is losing support the end of this month.

If I decide to proceed with the Developer program and paying the annual fee ... will the DMG builds from this computer become invalid, i.e. will I have to buy a newer Mac to continue building and signing deliverables?

Thanks again for your time.

It will only use 127.0.0.1 no matter which technique is used to executed it - even in the IDE debugger. And it changes the listener port.

This is not related to the trust execution system, so I recommend that you start a new thread for it. Tag it with Network so that I see it.

If I decide to proceed with the Developer program … will the DMG builds from this computer become invalid

In general, Developer ID signed products don’t expire. We say as much in Developer > Support > Certificates.

IMPORTANT That page says that Developer ID signed installer packages expire. That’s no longer correct. See this post.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"