[zeromq-dev] 0MQ + standard RPC

Matthew Long matthew.long at aptomar.com
Tue Nov 29 08:46:04 CET 2011

Hi :-)

On Nov 28, 2011, at 3:36 PM, Jakub Witkowski wrote:

> Hello,
> On 2011-11-28 14:00, Matthew Long wrote:
>> Right,
>> Conceptually a complete RPC system has:
>> i) An underlying connection or transport between two endpoints.
>> ii) A mechanism to handle reading and writing data from the connection
>> iii) A codec to transform data from a language-specific meaning to a common wire format
>> iv) An RPC system to do something with the messages
>> The points you mentioned are definitely part of this.  0MQ provides a very nice transport.  But since I am lazy, I'd like to not have to handle ii-iv myself.
> It's only my personal opinion, but after working with a bunch of "RPC" 
> libraries I came to conclusion that the only reason they realize points 
> II and IV is because the underlying transport is totally unsuited for 
> true RPC style communication and the wire protocol - and therefore point 
> III - has evolved well beyond initial assumptions into a "universal" 
> monstrosity that requires massive amount of careful work to produce and 
> parse back. Example: SOAP.

God no, I'd rather have to type the entire system with my nose :-/

No, the serialization is the easy part.  We're using protobufs for this.  The problem is that as I started building my system and I noticed that I had started basically rebuilding the entire RPC layer from the ground up.  The only difference was that the transport layer had changed and some of the primitives for the transport layer had a different set of semantics.  But everything above point (i) looked like wasted effort.

> On the other hand, making a "RPC-like" system, fully customized to your 
> application based on ZMQ is very simple; it more or less involves 
> copy-pasting the dealer/router pattern from the examples and  putting in 
> your business logic in. Toss in Google Protocol Buffers and suddenly you 
> have a full featured solution that is both easy to write and very, very 
> fast.

Copy and paste is not what I want to do.  I want to build a library so that I can separate my network communication from my business logic.  This is what a good RPC library really is.  Hence my original question.  

> In my humble opinion, you can write a remote call system using ZMQ with 
> *less effort* that would usually take integrating a RPC library with 
> your production code.

That may be, but it just seems that the code that I need to write is already there...  


> Jakub.
>> Those are the big pieces.  There are other considerations as well:
>> * Language-specific methods for handling the RPC call.  An idiomatic RPC layer for Java will be different than C++, for example.
>> * Asynchronous calls.  Often we'll want to have clients that can have several requests "in-flight" with asynchronous callbacks when particular calls complete.  I've been looking at XREQ/XREP for this (similar to the example in the guide).  In this case, REQ/REP is not really sufficient.
>> One thing I like about 0MQ is that with multipart messages we could use the transport layer to do more interesting routing than is normally the case with pure TCP.  This could actually be quite interesting in our domain.
>> Matt
>> On Nov 28, 2011, at 12:29 PM, Martin Sustrik wrote:
>>> Hi Matthew,
>>>> We will heavily leverage the 0MQ PUB/SUB functionality in most of the
>>>> communication layer that we are building.  The RPC portion is just
>>>> one piece of what we are doing -- in general it is simply going to be
>>>> the control channel.
>>>> If 0MQ is not a good fit for this piece, it is not a big deal, but I
>>>> had hoped that some of what you have built into 0MQ (such as durable
>>>> sockets) could help simplify some of the application layer.
>>> Ok. I see.
>>> Basically, RPC is the REQ/REP pattern.
>>> What you are missing for full-blown RPC solution is:
>>> 1. Serialisation -- this is delberately decoupled from 0MQ. As a
>>> consequence you can use any serialisation library you like.
>>> 2. Resending of the request is case the server dies. This should be and
>>> will be done inside of 0MQ, but is not done yet. However, it can be
>>> easily emulated on top of it.
>>> Anything I've missed?
>>> Martin
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

More information about the zeromq-dev mailing list