Faulty, hard-to-understand XPC behavior with remote methods that have a reply-block

Assume this over-simplified @protocol I'm using for my XPC-service:

@protocol MyMinimalProtocol <NSObject>
- (void)getStatusWithReply:(void (^ _Nullable)(NSDictionary * _Nonnull))reply;
@end

The Client side would then

NSXPCConnection *connection =  [[NSXPCConnection alloc] initWithMachServiceName:myServiceLabel options:NSXPCConnectionPrivileged];

connection.remoteObjectInterface = [NSXPCInterface interfaceWithProtocol:@protocol(MyMinimalProtocol)];
connection.interruptionHandler = ^{  NSLog(@"XPC: connection - interrupted"); };
connection.invalidationHandler = ^{ NSLog(@"XPC: connection - invalidated"); };
[connection resume];
[connection.remoteObjectProxy getStatusWithReply:^(NSDictionary * response) {
        NSLog(@"XPC service status received - %@", response);
}];

So far - so good. My XPC service receives the asynchronous call, schedules it's "status gathering operation" on internal background queue, and returns. Later - when information is available, my XPC service executes the reply-block then, on the remote calling side - I see the log line with the status, as expected.

BUT!!!

If I add another different code-block argument to the method e.g.

@protocol MyMinimalProtocol <NSObject>
- (void)getStatusWithReply:(void (^ _Nullable)(NSDictionary * _Nonnull))reply andFailureBlock:(void (^ _Nullable)(NSError * _Nonnull))failureBlock;
@end

Then all hell breaks loose. Both XPC service and the client crash with hilarious crash reasons I can't decipher.

Here's "Client side" caller crash (excerpt - forgive the methods are NOT the simplified ones above)

while on the "XPC Service" side, crashes like these:

I wonder if there's something inherently wrong with having two code-block arguments for an XPC remote method?

Another issue. The client XPC calls are asynchronous. They return immediately. The XPC service implementing the remote-call also returns immediately - and it executes the "reply block" far (a minute!) later, on another queue.

However, if the XPC service attempts to execute the code-block MORE THAN ONCE, then the client-side code-block is only called ONCE. rest of the executions look benign in the XPC-service side - but never happen on the calling (client) side.

Any idea why? can this be overcome?

Any thoughts/ideas/references to documentation will be greatly appreciated. I couldn't find any good source on this.

Thanks.

Accepted Reply

I wonder if there's something inherently wrong with having two code-block arguments for an XPC remote method?

Correct. NSXPCConnection requires that all protocol methods have at most one block parameter, that it be the last parameter, and that it has reply semantics. Adding extra block parameters isn’t supported.

However, if the XPC service attempts to execute the code-block MORE THAN ONCE

That’s also not supported.

You need to design your protocols with these restrictions in mind. For example:

  • You might have a single method that completes immediately with a request token.

  • And then have other methods that get the latest status for that token.

You can use an anonymous connection (via NSXPCListenerEndpoint) for this if you want.

You can also use NSProgress.

It’s also possible to set up bi-directional messaging, but that’s challenging because you want the server to be resilient in the face of a wedged client.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

  • Wow, THANK YOU! Could you please add few more details ?

    In what doc/TechNote - can I find these facts (what's supported and what isn't)? dev. on the basis of guess-work is slow and inefficient.Your suggested 2-point design allows POLLING on status, which is what I try to eliminate via two-way connection. Currently I maintain "named" service one-way + unnamed back-connection, but keeping the pair alive is tricky and error-prone.NSProgress? bi-directional? How!?
Add a Comment

Replies

I wonder if there's something inherently wrong with having two code-block arguments for an XPC remote method?

Correct. NSXPCConnection requires that all protocol methods have at most one block parameter, that it be the last parameter, and that it has reply semantics. Adding extra block parameters isn’t supported.

However, if the XPC service attempts to execute the code-block MORE THAN ONCE

That’s also not supported.

You need to design your protocols with these restrictions in mind. For example:

  • You might have a single method that completes immediately with a request token.

  • And then have other methods that get the latest status for that token.

You can use an anonymous connection (via NSXPCListenerEndpoint) for this if you want.

You can also use NSProgress.

It’s also possible to set up bi-directional messaging, but that’s challenging because you want the server to be resilient in the face of a wedged client.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

  • Wow, THANK YOU! Could you please add few more details ?

    In what doc/TechNote - can I find these facts (what's supported and what isn't)? dev. on the basis of guess-work is slow and inefficient.Your suggested 2-point design allows POLLING on status, which is what I try to eliminate via two-way connection. Currently I maintain "named" service one-way + unnamed back-connection, but keeping the pair alive is tricky and error-prone.NSProgress? bi-directional? How!?
Add a Comment

Just a reminder: Please reply in a reply. If you reply in the comments, I’m not notified. See tip 5 in Quinn’s Top Ten DevForums Tips.


In what doc/TechNote - can I find these facts (what's supported and what isn't)?

The current official documentation for NSXPCConnection leaves a lot to be desired )-: Please do file a bug about that.

The only doc I could find that covers this is in the Documentation Archive, namely Daemons and Services Programming Guide > Creating XPC Services > Using the Service > Designing an Interface [1], which says:

Because communication over XPC is asynchronous, all methods in the protocol must have a return type of void. If you need to return data, you can define a reply block like this … A method can have only one reply block.

The rest of that doc is also well worth a read.

However, my all-time favourite documentation for this is WWDC 2012 Session 241 Cocoa Interprocess Communication with XPC. That’s long since been removed from the Apple Developer website but, if you have a copy sequestered away somewhere, it’s well worth your time.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Which is the Apple documentation equivalent of Beware of the Leopard. (-:

Thanks again.

Indeed I missed the sentence "A method can have only one reply block." in the old archived documentation - but that same sentence also says:

A method can have only one reply block. However, because connections are bidirectional, the XPC service helper can also reply by calling methods in the interface provided by the main application, if desired.

Which brings me to the original question -- HOW do I do that? My "players" aren't an App and its helper-service, but rather independent, resilient system daemons and global agents that need communicate, in all directions. The dual XPC connection I'm maintaining now, is just cumbersome. Each "product shutdown" flow is an ugly nightmare, and recovering from one-side-crash looks different on either sides - which is ugly too. Bidirectional XPC connections look like the right thing to me.

I found and downloaded the recommended 2012 WWDC session from here: https://archive.org/download/wwdc-2012-sessions and watched it carefully twice - but there too - it is only HINTED for a split second, that the connection is bidirectional and both sides can send messages - and immediately the hint is removed from screen, and replaced by that single reply-block technique.

There is a tiny gap here I need to bridge. I already have a live NSXPCConnection, both sides agree on the same protocol, the "client side" obtains a "remote proxy" and the "server side" exports an object and assigns them to accepted incoming connection. Then client sends messages to the service.

How can the client-side create an "exported object"? can it also assign an NSXPCInterface and exported-object to its NSXPCConnection when it connects to the service? and how would the service get the proxy if it wants to call-back to the client?

If the docs say NSXPCConnection is bi-directional, SOMETHING must be said somewhere regarding the use of this feature.

As in your hinted novel - I think I'll just go on and open the "disused lavatory" carrying that "Beware of the Leopard" sign, to find the demolition order for my house :(

One things keeps me uneasy. In the full decade of NSXPCConnection, hasn't anyone need bi-directional IPC? Why is it only I'm looking for ways to do it? Design-wise, Is it something wrong with my ideas?

Setting up bidirectional XPC is ridiculously straightforward, although it took me years to realise it was this easy! (-: In the classic XPC workflow, the client sets up a connection like so:

connection.remoteObjectInterface = NSXPCInterface(with: MyClientToServerProtocol.self)

and the server like so:

self.xpcConnection.exportedInterface = NSXPCInterface(with: MyClientToServerProtocol.self)
self.xpcConnection.exportedObject = self

For the server to talk back to the client, you need to do the reverse:

  • Define MyServerToClientProtocol.

  • On the server, set remoteObjectInterface to it.

  • On the client, set exportedInterface to it and exportedObject to an object that implements it.

There are other options though:

  • You can create an anonymous XPC listener on the client and pass its endpoint over to the server. The server can then connect back to the client.

  • You can create an object that implements an XPC compatible protocol on the client and pass that to the server. The server gets a proxy for that object and can message it.

  • NSXPCConnection has NSProgress integration. [Now where is that documented… Oh, here you go…] See the NSXPCConnection support for discrete NSProgress section of Foundation Release Notes for macOS 10.13 and iOS 11.

My "players" aren't an App and its helper-service, but rather independent, resilient system daemons and global agents that need communicate, in all directions.

Endpoints can help with this. They allow you to set up connection graphs more complex than a simple N-clients-to-1-server star. For example, see the XPC Rendezvous section of XPC and App-to-App Communication.

Design-wise, Is it something wrong with my ideas?

No, but you do have to be careful when crossing trust domains. Imagine you have a server that sends a message to a client. If the server is a daemon and the client is an agent, think about what happens if the client simply stops processing requests. The requests back up in the receiver’s Mach port, but that fills up pretty quickly (the standard queue length is 5 IIRC). After that they start stacking up in the server’s address space [1]. This represents a potential denial of service attack on the server.

This problem is particularly bad on iOS because of the way that iOS suspends apps when they’re in the background. Fortunately, there aren’t many third-party folks doing XPC on iOS [2].

One solution is to issue a -scheduleSendBarrierBlock: after your server-to-client request and then refuse to issue any more until the barrier block is called.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Assuming you’re using async messaging. If you use -synchronousRemoteObjectProxyWithErrorHandler: then the calling thread blocks, which would be Bad™.

[2] AFAIK the only third-party XPC support on iOS is for file providers.

  • I bow to DTS god. Thank you so much. As a person who bought and read the full original "Inside Macintosh" series, I got spoiled early on, believing in Apple Documentation, and its completeness of description. Each subsystem, from "conceptual" to "programming tasks" to "reference". Nowadays I simply can't find the information anymore, and much of the remaining information is deemed "deprecated", "retired", "archives" and other words describing neglect. Thank god YOU still remember.

Add a Comment