App won't start after package install without reboot

For many versions of macOS I have shipped an application, EndpointService. EndpointService is installed in /Library/Application Support/Identity Finder. It runs as root, and is owned by root:wheel. It is controlled by a LaunchDaemon plist. After install, the newly installed version has always run without a problem without a rebooting the machine. Recently (I think starting in Mojave), EndpointService will not start until the machine is rebooted, then after that it runs normally. Without a reboot, running launchctl load fails to start the app, and even just doing a version check from the command line results in "Killed: 9". Is this a SIP problem? Is a reboot now required?


Here is the com.identityfinder.launchdaemon.plist file:


<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>EnvironmentVariables</key>

<dict>

<key>LANG</key>

<string>C</string>

</dict>

<key>AbandonProcessGroup</key>

<true/>

<key>KeepAlive</key>

<true/>

<key>Label</key>

<string>com.identityfinder.launchdaemon</string>

<key>ProgramArguments</key>

<array>

<string>/Library/Application Support/Identity Finder/EndpointService</string>

<string>--launchd</string>

</array>

<key>StartInterval</key>

<integer>300</integer>

</dict>

</plist>


Here's the listing:


$ ls -l

total 133136

-rwxr-xr-x@ 1 root wheel 36519424 Jan 10 10:54 EndpointService


If I even try to just get the version, it won't start:


$ /Library/Application\ Support/Identity\ Finder/EndpointService --version

Killed: 9

Up vote post of michaeldomino Down vote post of michaeldomino
1.4k views
Posted by
michaeldomino
Reply
Add a Comment

Replies

even just doing a version check from the command line results in [

SIGKILL
].

Do you get a crash report in that case? If so, please post it here (use the 

<>
button to format it as code).

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware 

let myEmail = "eskimo" + "1" + "@apple.com"
Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment

Yes, a log is there, thank you for the suggestion! I see the error "EXC_CRASH (Code Signature Invalid)". So my question is, why would this error be cured by restarting?


Process:               EndpointService [6064]
Path:                  /Library/Application Support/Identity Finder/EndpointService
Identifier:            EndpointService
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        launchd [1]
Responsible:           EndpointService [6064]
User ID:               0


Date/Time:             2020-01-10 10:40:37.149 -0500
OS Version:            Mac OS X 10.14.6 (18G2016)
Report Version:        12
Bridge OS Version:     4.2 (17P52548a)
Anonymous UUID:        22B540BE-F410-DFF0-2327-BE12160751EF


Sleep/Wake UUID:       16F3E38B-6D6F-4DAE-85F0-7AD5568471D5


Time Awake Since Boot: 45000 seconds
Time Since Wake:       6200 seconds


System Integrity Protection: enabled


Crashed Thread:        Unknown


Exception Type:        EXC_CRASH (Code Signature Invalid)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY


Termination Signal:    Terminated: 15
Termination Reason:    Namespace SIGNAL, Code 0xf
Terminating Process:   launchd [1]


kernel messages:


Backtrace not available


Unknown thread crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000000000000  rcx: 0x0000000000000000  rdx: 0x0000000000000000
  rdi: 0x0000000000000000  rsi: 0x0000000000000000  rbp: 0x0000000000000000  rsp: 0x00007ffeedfa2d60
   r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000000
  r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000000000000000  r15: 0x0000000000000000
  rip: 0x000000010a49a000  rfl: 0x0000000000000200  cr2: 0x0000000000000000
 
Logical CPU:     0
Error Code:      0x00000000
Trap Number:     0




Binary images description not available




External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 838407
    thread_create: 0
    thread_set_state: 13484
Posted by
michaeldomino
Up vote reply of michaeldomino Down vote reply of michaeldomino
Add a Comment

This sounds very much like a code signature caching issue. Apple platforms cache information about a Mach-O image’s code signature in the kernel and do not invalidate that cache when you modify the underlying file. For this reason, it’s critical to never modify an executable. Rather, if you’re writing an installer, you should write the new contents to a new file and then move that file into place (using 

rename
). Failure to do this can result in weird problems exactly like the one you describe.

So, are you installing this daemon’s executable with the Apple installer? Or an installer you wrote? And if it’s the latter, does it modify the executable in place?

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware 

let myEmail = "eskimo" + "1" + "@apple.com"
Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo

  • I am having same issue. I am using shell script to upgrade the app. Is this rename a command? How to use it with shell script?

    —
    ManojPattanaik
Add a Comment

I am having same issue. I am using shell script to upgrade the app. Is this rename a command? How to use it with shell script?

Posted by
ManojPattanaik
Up vote reply of ManojPattanaik Down vote reply of ManojPattanaik
Add a Comment

I am using shell script to upgrade the app. Is this rename a command?

Rename is a system call. The shell equivalent is mv.

Here’s a quick diagnostic you can use:

  1. Before the upgrade, do an an ls -i on the executable.

  2. Then do the upgrade.

  3. Then do an ls -i again.

Does the inode number change?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment

This sounds very much like a code signature caching issue.

I wrote an article, Updating Mac Software, that covers this issue in more detail.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment