[zeromq-dev] multiplexer pattern

MinRK benjaminrk at gmail.com
Thu Sep 22 02:04:51 CEST 2011

On Sun, Sep 18, 2011 at 00:58, Martin Sustrik <sustrik at 250bpm.com> wrote:

> Hi Benjamin,
> The request/reply is moving to what it really is: A way for clients to
> access a set of homogenous and stateless services, providing a way to
> load-balance and handle failure in distributed systems.
> If you need multiple types of services, each provided by a different set of
> workers, you should create multiple topologies.
> Now, there are many use cases that don't fit into request/reply pattern --
> in most cases these involve statefull workers. For such use cases there's
> the generic ROUTER socket in 4.0.
> The idea is that ROUTER socket allows you to do basically anything (send
> messages in any order, to any peer, you get notification about connects and
> disconnects etc.) and thus can be used to create new messaging patterns on
> top of 0MQ.
> If it turns out that the pattern is generally useful and well-defined it
> can then be merged into 0MQ itself. To create such a well-defined pattern
> you should think about how the pattern behaves in the face of overload
> (block? drop messages? retry?), how it deals with rogue clients (how to
> handle DoS attacks?), how it scales (can we add more intermediary nodes to
> the topology without breaking the semantics?) etc.
>  It would seem that this extremely basic pattern in 2.1 has become quite
>> complicated after the removal of identities.  I imagine that I will just
>> have to reimplement identity-routing myself on top of the new ROUTER
>> socket, with a lookup table in the MUX matching the engine-specified
>> identities to the real routing prefixes, and add extra handshaking to
>> the connection process to get this information everywhere it needs to be.
> Yes. It's going to be somewhat more painful, but AFAICS it boils down to:
> 1. Send a identity message manually after the connect.
> 2. On the server side, store the identity & connection ID in a map.
> 3. When request arrives at the server, check the map and forward the
> message to the right connection.

Yes, this is *almost* straightforward.  One problem with the SNDCMD
implementation is that I cannot handle connection events separately from
actual messages.  Now, I think it's perfectly fine that the connection info
messages are coming in the same channel, but what doesn't work well is that
I have to receive the first message part before making the decision about
what to do with it.  It means always putting branches one step later than
they would naturally go.

Would it be possible to have something like an IDENTITY that gets added to
the initial SNDCMD message, that I can specify?  That would make part of
this a good deal simpler, even if I have to do the mapping myself, or a
mechanism to determine if the message waiting to be received in a CMD?

I would really like to be able to write a recv_commands(s) method that would
recv and handle any *commands* that are waiting, but not real messages.
 This is impossible to implement if CMD can only be checked after receiving
the message, which forces command-handling into the recv routine, which
doesn't make a lot of sense.

There are significant advantages of the IDENTITY design - I can easily scale
the number of intermediate multiplexer devices with no changes to the code
beyond an extra 'connect'.  Now, that's not true, because I have to go
through some variation of a handshake process with each MUX.

Essentially, I had a model where one client talks to one service.  I could
then insert the MUX device, and I had m-clients taking to n-services by name
via p-MUXes, for perfectly arbitrary values of m,n,p, without any change to
the code - the clients and services have no idea whether there is a MUX in
place or not.  That is simply no longer possible, as each component must now
know much more information about the state of the system.

I recognize that there are many more things that can be done with the
generic 4.0 ROUTER, but it does not seem to be true that it can replicate
the 2.1 ROUTER cleanly or efficiently.  Hopefully we can figure something
out by the time we have to start using 4.0.


> Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110921/f0fc28ca/attachment.htm>

More information about the zeromq-dev mailing list