[zeromq-dev] Publish / Subscribe vs Multicast

gonzalo diethelm gdiethelm at dcv.cl
Fri Feb 12 21:31:15 CET 2010


Hi Brian,

> I too am just getting started with zeormq and our system (see my
> recent email to the list) has some overlap with yours.  Ours is even
> more complex perhaps.

It looks like your usecase is more of a (synchronous) RPC model, right?

> My thought is to use use a PUB/SUB model with topics for each worker.
> When a worker attaches, it would send a presence or registration
> message to the central messag hub.  That hub would assign a topic to
> the worker.  From then on, the worker would subscribe to that topic
> and the scehduler (application level) would append that topic to send
> tasks to the worker.

This means you would have to know beforehand all the possible topics,
right? What if all the workers subscribed to one given topic, crash?

> Yes, these are exactly the questions I am struggling with.  Why don't
> you join in on the other threads going on to continue this discussion.
>  I can see how to handle new workers joining in (they send a
> registration msg and are allocated a topic), but i am struggling with
> how to handle workers going away.  There needs to be some way for the
> application level scheduler to discover that a worker is dead and
> should not be allocated tasks.

It seems so.

> Yes, I am definitely thinking of having application level queues and
> scheduling logic.  And the same questions to apply.

I think it makes more sense to expose zmq's queues so that they can be
managed from your code.

> One more comment:  the current design of zeromq seems to be focused on
> messages as packets of information.  But in the usage cases you and I
> are describing, our messages are "actions" more like you see in RPC
> systems.  But, the problem with  RPC systems is that they are not
> asynchronous or fault tolerant enough.

I am trying to avoid putting myself in an RPC mind frame... That's why I
keep talking about an (asynchronous) pipeline.

> Very timely post!

Thanks!

Gonzalo



----------------------------------------- 
Declaración de confidencialidad: Este Mensaje esta destinado para
el uso de la o las personas o entidades a quien ha sido dirigido y
puede contener información reservada y confidencial que no puede
ser divulgada, difundida, ni aprovechada en forma alguna. El uso no
autorizado de la información contenida en este correo podrá ser
sancionado de conformidad con la ley chilena. 
Si usted ha recibido este correo electrónico por error, le pedimos
eliminarlo junto con los archivos adjuntos y avisar inmediatamente
al remitente, respondiendo este mensaje. 

"Before printing this e-mail think if is really necesary".
Disclosure: This Message is to be used by the individual,
individuals or entities that it is addressed to and may include
private and confidential information that may not be disclosed,
made public nor used in any way at all. Unauthorized use of the
information in this electronic mail message may be subject to the
penalties set forth by Chilean law. 
If you have received this electronic mail message in error, we ask
you to destroy the message and its attached file(s) and to
immediately notify the sender by answering this message. 




More information about the zeromq-dev mailing list