IPC always has some per-message overhead, so you get more throughput if you batch your messages. However, batching introduces latency. Whether that latency matters depends on how “real” your real-time goal is.
The aim it to try to get the user to believe the processing is happening in the app, instead of the background service, so any user-perceived lag would be missing the goal.
The key word there is "user perceived lag". The actual activity volume is some much higher an faster than the users ability to process data that any other latency is basically irrelevant.
What you're REALLY talking about here is that you'd like to create an animation that is:
-Fast enough that the user thinks things look really speedy.
-Slow enough that the user has a sense of what's going on.
-"Truth-y" enough that the contents generally correspond to whatever the user thinks is actually happening.
The key is that the word "animation" is critical here. This is NOT about "real time data", it's about presenting an attractive experience to the user. You're thinking about this in terms of XPC/IPC overhead, but the reality is:
-XPC/IPC latency is FAR smaller than screen latency.
-The users ability to read/process anything you put on the screen is FAR slower than the screen update latency.
There's no point trying to get your app more data faster. XPC can easily send data far faster than the screen can show it, just like the screen can show it far faster than the user could read it.
My actual suggestion here is that you START by figuring out what you want this to look like, THEN decide what data to back fill into that experience. I would quite literally start by experimenting with Timer's and fake data to figure out what looks attractive/useful/etc.
I was curious enough that I actually ended up mocking this up myself using a timer, a text field, and some simple numeric formatting to fake something short and vaguely textual. Basically, ~10 updates/sec was very fast but readable, ~30/s was very fast/blurry, 120/s was a vaguely numeric blur. Do the same tests yourself and I believe what you'll discover is that the interface that actually looks good is SO slow relative to the volume of data the ES client COULD provide that any kind of performance/bandwidth concern is totally irrelevant. The best approach here isn't about the ES Client sending "all" it's activity to the app, it's about the ES Client cherry picking the most "interesting" actions and then sending that to the client at regular intervals.
I understand what you're saying about displaying useful information to the user. I ran the Red Canary Endpoint Security client a while back to see how they handled the performance issues around the Endpoint Security framework, and noticed that their GUI interface isn't dealing particularily well with the high volume
Note that, in my experience, many of these issues are caused by the GUI being drowned in noise, NOT because the GUI didn't have enough data to look at. That is, GUI ends up looking slow and clunky because it's wasting its time deciding what to show "next" instead of ACTUALLY showing something. Getting the data to the app "faster" just slows the interface down more.
__
Kevin Elliott
DTS Engineer, CoreOS/Hardware