[zeromq-dev] 0MQ + standard RPC

MinRK benjaminrk at gmail.com
Tue Nov 29 19:58:49 CET 2011


On Tue, Nov 29, 2011 at 00:50, Matthew Long <matthew.long at aptomar.com>wrote:

> As a short followup -- I just looked at PyZMQ and it looks like an
> excellent "pythonic" binding.  In particular, it handles asynchronous calls
> and queueing calls so that they can be sent via a single socket thread.
>

Glad you like it!  The tornado IOLoop/ZMQStream stuff in zmq.eventloop
should make writing a Python RPC server pretty straightforward.  The model
we use in IPython.parallel is essentially Async RPC-ish, and we use JSON to
serialize by default, but allow drop-in replacement of other serialization
methods for performance/language compatibility. I've had good experience
with msgpack, and I believe protobuf works as well.

-MinRK


> Unfortunately, I am not doing anything in Python at the moment.
>
> Cheers,
>
> Matt
>
> On Nov 29, 2011, at 9:29 AM, Matthew Long wrote:
>
> > Hi Pieter,
> >
> > On Nov 28, 2011, at 10:03 PM, Pieter Hintjens wrote:
> >
> >> On Mon, Nov 28, 2011 at 8:36 AM, Jakub Witkowski <jpw at jabster.pl>
> wrote:
> >>
> >>> On the other hand, making a "RPC-like" system, fully customized to your
> >>> application based on ZMQ is very simple; it more or less involves
> >>> copy-pasting the dealer/router pattern from the examples and  putting
> in
> >>> your business logic in. Toss in Google Protocol Buffers and suddenly
> you
> >>> have a full featured solution that is both easy to write and very, very
> >>> fast.
> >>
> >> Yes, this is what I'd recommend as well. Google protobufs give you
> >> language portability, are fast, and easy to use. There are various RPC
> >> examples in the Guide you can start with, see Ch3 and Ch4. I'd suggest
> >> either the Majordomo or Freelance patterns.
> >
> > Yes, those are a good start.  We already use protobufs for the IDL,
> service definitions and serialization.
> >
> > Where these examples start breaking down when I try to bring them into
> my existing frameworks:
> > 1) Sockets are not thread safe so we need to build scaffolding to
> protect the socket.  When threads are expensive like in C/C++/etc it makes
> sense to do this via inproc:// connections, but in other languages (such as
> Go) where go routines can be multiplexed onto any number of running threads
> (by the runtime) this is an issue.
> > 2) Clients often want asynchronous messages and callbacks.  That is, the
> client might send requests 1, 2, 3, 4, 5, but get back 5, 3, 4, 1, 2 -- and
> we need to pair the request context with the correct response.
> > 3) Go is a typesafe language, but we want to be able to do the type
> conversion when we serialize and deserialize messages.
> >
> > All three are already handled in the RPC layer -- the fact that I was
> reimplementing most of this is what prompted my post to the list.
> >
> > What I am hearing is there are no existing tools to do this and I should
> implement something myself.
> >
> > It may be that with the appropriate surgery I can fit 0MQ in with
> existing frameworks -- but this is something that the 0MQ team might want
> to consider addressing at some point.  While there are language bindings in
> a ton of languages, a strict port of the C api probably won't be idiomatic
> in the target language.  If 0MQ fit in easily to the right layer of Go or
> Java or Python it would be a clear win.  I could use 0MQ with the native
> RPC system and also implement neat patterns on top of it for PUB/SUB or
> whatever.  As it stands it is a little bit of a mixed win for us.
> >
> > If I do figure out how to cleanly add 0MQ to various RPC systems, I will
> post what I did to the list.
> >
> > Matt
> >
> >>
> >> -Pieter
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20111129/f06223a2/attachment.htm>


More information about the zeromq-dev mailing list