[zeromq-dev] Publish / Subscribe vs Multicast

Brian Granger ellisonbg at gmail.com
Fri Feb 12 21:41:26 CET 2010


>> I too am just getting started with zeormq and our system (see my
>> recent email to the list) has some overlap with yours.  Ours is even
>> more complex perhaps.
> It looks like your usecase is more of a (synchronous) RPC model, right?

Not really.  Everything is extremely async in our system (in terms of
the evolution
of he system with time).  While some types of messages sort of have a
request/reply form, it is still very asynch.  Also some types of request can
generate entire streams of message replies.

Our current version is implemented on top of an RPC protocol.  BUT, we have
basically has to implement our own message system using that RPC protocol.

>> My thought is to use use a PUB/SUB model with topics for each worker.
>> When a worker attaches, it would send a presence or registration
>> message to the central messag hub.  That hub would assign a topic to
>> the worker.  From then on, the worker would subscribe to that topic
>> and the scehduler (application level) would append that topic to send
>> tasks to the worker.
> This means you would have to know beforehand all the possible topics,
> right? What if all the workers subscribed to one given topic, crash?

These are the questions i am trying to answer.  It would also be possible
that the message hub could allocate a new, unique topic when a worker
connects.  This would enable it to be on the fly.

> I think it makes more sense to expose zmq's queues so that they can be
> managed from your code.

Possibly.  It does seem a bit odd to have to maintain my own queues in
a message system.  But, on the other hand our scheduling algorithms
require reasonably complex directed acyclic graphs of queues.  Not sure
I would want to use 0MQ for all of that.

>> One more comment:  the current design of zeromq seems to be focused on
>> messages as packets of information.  But in the usage cases you and I
>> are describing, our messages are "actions" more like you see in RPC
>> systems.  But, the problem with  RPC systems is that they are not
>> asynchronous or fault tolerant enough.
> I am trying to avoid putting myself in an RPC mind frame... That's why I
> keep talking about an (asynchronous) pipeline.

Yes, me too.  We have learned the hardway that the RPC approach does not
fit well with the types of things we are going.



>> Very timely post!
> Thanks!
> Gonzalo
> -----------------------------------------
> Declaración de confidencialidad: Este Mensaje esta destinado para
> el uso de la o las personas o entidades a quien ha sido dirigido y
> puede contener información reservada y confidencial que no puede
> ser divulgada, difundida, ni aprovechada en forma alguna. El uso no
> autorizado de la información contenida en este correo podrá ser
> sancionado de conformidad con la ley chilena.
> Si usted ha recibido este correo electrónico por error, le pedimos
> eliminarlo junto con los archivos adjuntos y avisar inmediatamente
> al remitente, respondiendo este mensaje.
> "Before printing this e-mail think if is really necesary".
> Disclosure: This Message is to be used by the individual,
> individuals or entities that it is addressed to and may include
> private and confidential information that may not be disclosed,
> made public nor used in any way at all. Unauthorized use of the
> information in this electronic mail message may be subject to the
> penalties set forth by Chilean law.
> If you have received this electronic mail message in error, we ask
> you to destroy the message and its attached file(s) and to
> immediately notify the sender by answering this message.

Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com

More information about the zeromq-dev mailing list