LaunchAgent without a GUI session?

This question came from https://developer.apple.com/forums/thread/695826, where I saw crashes in AppKit if called without a GUI session. What troubles me from there is that our code is registered as a LaunchAgent (under /Library/LaunchAgents), and I was under the impression that a LaunchAgent only runs if a user logs into a GUI session. I tried at least ssh-only sessions and didn't see it launch automatically (I had to manually launch it through ssh to reproduce the crash). But the fact that we see thousands of crash reports coming from a few devices means somehow our LaunchAgent is trying to launch itself automatically & repeatedly on these devices, while there is no GUI session so it keeps crashing.

So, maybe there is a legit way to reproduce the scenario, to launch a LaunchAgent without a GUI session that I'm not aware of?

For a bunch of backstory on this, read Technote 2083 Daemons and Agents. Some of the fine details have drifted a bit since I last updated it (2007!) but the overall concepts still hold up well.

The system always loads a launchd daemon into the global context; that’s what makes it a daemon. The story for launchd agents is more nuanced. The system loads your agent in all contexts that match the LimitLoadToSessionType property in your launchd.plist. See the launchd.plist man page for info on that property and the launchctl man page for a list of session types.

If you don’t specify a session type then the system assumes a default of Aqua and your agent should only run in GUI contexts. I presume that’s how your agent is configured, in which case it’s a mystery how your agent is hitting the AppKit panic that you discussed in your other thread. It’s possible that this is Just a Bug™, but it’s also possible that the user, or some software they’ve run, has monkeyed with your config in a way that triggers this.

The only way to know for sure is to capture info from a Mac that’s exhibiting the problem. I presume that you’re not in touch with any of these users?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks for the pointer to the Technote Eskimo! It was good reading.

I presume that you’re not in touch with any of these users?

That is unfortunately true. From crash logs alone we currently have no means to backtrack to the specific users. We'll have to wait for the users to come contacting us instead. Given that the crash is likely only happening without a GUI session it's likely they would not notice it for quite a while (or maybe not at all).

For now I think my interpretation of your answer is that from the OS point of view, there is no official way (at least no known one) to be able to launch an agent under the Aqua type (and yes our launchd.plist doesn't specify LimitLoadToSessionType) while denying it access to the graphic context. So the users must have somehow either "monkeyed" with the system, or like you suggested, with our config.

Assuming our hypothesis is true and there is no good option to prevent end users from doing what they are doing, currently our team has come up with an idea of detecting this situation (no GUI context) during our agent's startup in hoping to skip the entire AppKit entry call. Specifically we are thinking of calling CGWindowListCopyWindowInfo(kCGWindowListOptionAll, kCGNullWindowID) before we engage all UI related stuff. We verified that if we call this without graphic context we would get nil while otherwise we would get an array with something, which serves as an indicator to us. Does this sound like a practical idea or is there perhaps anything we overlooked?

So the users must have somehow either "monkeyed" with the system, or like you suggested, with our config.

Or there’s a bug in the OS )-:

Or there’s a bug in your code that’s ‘breaking’ AppKit’s state [1].

I presume that you’re not in touch with any of these users?

That is unfortunately true.

Yeah, I thought as much.

I regularly help developers in situations like this. My general advice is that you add code to detect the problem and then specifically prompt the user to trigger a sysdiagnose and then get in touch with you. Depending on your user base, you might want to enable that feature only for your beta testers.

Does this sound like a practical idea … ?

It would certainly be worth a shot. I can’t guarantee that it’ll reliably detect this issue (per the comments at the start of this post) but there’s a good chance it will.

You can also explicitly check for GUI access using SessionGetInfo to get the sessionHasGraphicAccess flag.

It might be worthwhile doing both to see if there’s any disparity.

Finally, can you post a full crash report? Your other thread didn’t have that and, now that I’ve dug into this in a bit more detail, I like to take a proper look at one.

See Posting a Crash Report for advice on how to post this.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] For example, you could have code that over-releasing a Mach port name. Most of the time the port name is invalid and so this results in an error that’s then ignore. However, if you manage to hit a port name that’s used by AppKit to talk to CG, you’d see an issue like this. But that’s just a wild theory, not something based on any evidence.

My point here is that problems like this can be really obscure.

SessionGetInfo sounds like a great idea! I'll forward it to our team.

can you post a full crash report?

Yes I can help with that. We already have a radar ticket on the original AppKit crash. Let me ask my colleague who filed it about the number and get back here. It's around the holidays though I might not be able to reach him soon.

And in the likely case I don't post any feedback until New Year just want to wish you Happy Holidays Eskimo (and of course to anyone who stumbles on this post around this time of year), your help and advice are always appreciated!

Radar ticket for this - FB9800999

Radar ticket for this - FB9800999

Thanks.

I see two crash reports attached to that. In one the parent process is bash with a responsible process of sshd, suggesting that this is your attempt to reproduce the problem in-house. The other one, output_18765164.log, is… well… weird looking. Was that generated by the standard Apple crash reporter?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Happy New Year eskimo! The crash logs we collect from our customers are generated by PLCrashReporter I believe. It somehow yanks the system symbols out (sounds like a PLCrashReporter thing) and we don't have a convenient way to re-symbolicate that part. That said I'm fairly confident if we would symbolicate output_18765164.log it would produce exactly the same crashed stack as the bash one (apart from the fact of different parent processes).

One more detail we recently figured out - we noticed that currently our crash reporter has a restriction built in our code that it's only able to properly process & upload crash logs if it's running with root access. Since we are getting these crash logs, it proves those processes have been launching as the root user, which seems to conflict with the fact that they are (at least supposed to be) registered as Launch Agent and should be running as the current logged in user? Again to me it still doesn't pinpoint the issue any further than our current speculation - it could still be a system bug/software bug/user tampering, but I thought to mention this new detail.

Happy New Year … !

Likewise!

The crash logs we collect from our customers are generated by PLCrashReporter I believe.

I’m not a big fan of third-party crash reporters. For an explanation as to why, see my Implementing Your Own Crash Reporter post.

Since we are getting these crash logs, it proves those processes have been launching as the root user, which seems to conflict with the fact that they are (at least supposed to be) registered as Launch Agent and should be running as the current logged in user?

It is possible for an agent to be run as root. Indeed, there are two ways that this can happen:

  • An agent configured to run at login time (with LoginWindow in LimitLoadToSessionType) will run as root.

  • If the user enables the root account, they can log in as root via the GUI. This is a full GUI login session, so standard GUI launchd agents (session type Aqua) will run, but they’ll run as root.

The first point is probably irrelevant to your situation. The second point is more interesting. In this case the agent should be running in a GUI login session and thus can talk to the window server just fine, it’s just that it’s running as root. It’s possible that’s triggering some other oddity…

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

LaunchAgent without a GUI session?
 
 
Q