[zeromq-dev] multiplexer pattern

Brian Granger ellisonbg at gmail.com
Wed Sep 21 02:30:40 CEST 2011


Martin,

On Sun, Sep 18, 2011 at 12:58 AM, Martin Sustrik <sustrik at 250bpm.com> wrote:
> Hi Benjamin,
>
> The request/reply is moving to what it really is: A way for clients to
> access a set of homogenous and stateless services, providing a way to
> load-balance and handle failure in distributed systems.

But the request reply pattern works applies equally well to stateful
services as well.

> If you need multiple types of services, each provided by a different set
> of workers, you should create multiple topologies.

Are you suggesting that the central router logic be accomplished with
a ZMQ socket per service type?  That would be a huge step backwards.
The usage case that Min is referring to could have dozens to hundreds
or even thousands of "service types" that we need to route messages
to.  And there is *always* only one instance of each service type.
Requiring each to be managed using a separate ZMQ socket is hugely
inefficient.

If this usage case cannot be addressed in 4.0 with a single socket
that can route, then we need to reconsider some of the design.

> Now, there are many use cases that don't fit into request/reply pattern
> -- in most cases these involve statefull workers. For such use cases
> there's the generic ROUTER socket in 4.0.
>
> The idea is that ROUTER socket allows you to do basically anything (send
> messages in any order, to any peer, you get notification about connects
> and disconnects etc.) and thus can be used to create new messaging
> patterns on top of 0MQ.
>
> If it turns out that the pattern is generally useful and well-defined it
> can then be merged into 0MQ itself. To create such a well-defined
> pattern you should think about how the pattern behaves in the face of
> overload (block? drop messages? retry?), how it deals with rogue clients
> (how to handle DoS attacks?), how it scales (can we add more
> intermediary nodes to the topology without breaking the semantics?) etc.
>
>> It would seem that this extremely basic pattern in 2.1 has become quite
>> complicated after the removal of identities.  I imagine that I will just
>> have to reimplement identity-routing myself on top of the new ROUTER
>> socket, with a lookup table in the MUX matching the engine-specified
>> identities to the real routing prefixes, and add extra handshaking to
>> the connection process to get this information everywhere it needs to be.
>
> Yes. It's going to be somewhat more painful, but AFAICS it boils down to:
>
> 1. Send a identity message manually after the connect.
> 2. On the server side, store the identity & connection ID in a map.
> 3. When request arrives at the server, check the map and forward the
> message to the right connection.
>
> Martin
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com



More information about the zeromq-dev mailing list