[zeromq-dev] multiplexer pattern

[Jakub Witkowski]

Hello all,

2011/9/21 Brian Granger <ellisonbg at gmail.com>:
> Martin,
>
> On Sun, Sep 18, 2011 at 12:58 AM, Martin Sustrik <sustrik at 250bpm.com> wrote:
>> Hi Benjamin,
>>
>> The request/reply is moving to what it really is: A way for clients to
>> access a set of homogenous and stateless services, providing a way to
>> load-balance and handle failure in distributed systems.
>
> But the request reply pattern works applies equally well to stateful
> services as well.
>

Request/reply is really limited by it's forced synchronous nature.


>> If you need multiple types of services, each provided by a different set
>> of workers, you should create multiple topologies.
>
> Are you suggesting that the central router logic be accomplished with
> a ZMQ socket per service type?  That would be a huge step backwards.
> The usage case that Min is referring to could have dozens to hundreds
> or even thousands of "service types" that we need to route messages
> to.  And there is *always* only one instance of each service type.
> Requiring each to be managed using a separate ZMQ socket is hugely
> inefficient.
>

I have a use case where not only there are N named services, the
number is indeterminate and dynamic. I am writing a MMO-like game and
while the bulk of the data will probably be moved through anonymous
(and dynamically established) pub/sub sockets, there is still the need
for clients to talk to instance servers through a centralized proxy.
Things like join/leave requests, global user interaction (instant
messages, friendship and party requests, etc), inventory management,
etc, should go through a central router, not only to decouple both
client and server code from resource location logic, but also to
provide a convenient point for packet inspection.


> If this usage case cannot be addressed in 4.0 with a single socket
> that can route, then we need to reconsider some of the design.
>
>> Now, there are many use cases that don't fit into request/reply pattern
>> -- in most cases these involve statefull workers. For such use cases
>> there's the generic ROUTER socket in 4.0.
>>
>> The idea is that ROUTER socket allows you to do basically anything (send
>> messages in any order, to any peer, you get notification about connects
>> and disconnects etc.) and thus can be used to create new messaging
>> patterns on top of 0MQ.
>>
>> If it turns out that the pattern is generally useful and well-defined it
>> can then be merged into 0MQ itself. To create such a well-defined
>> pattern you should think about how the pattern behaves in the face of
>> overload (block? drop messages? retry?), how it deals with rogue clients
>> (how to handle DoS attacks?), how it scales (can we add more
>> intermediary nodes to the topology without breaking the semantics?) etc.
>>
>>> It would seem that this extremely basic pattern in 2.1 has become quite
>>> complicated after the removal of identities.  I imagine that I will just
>>> have to reimplement identity-routing myself on top of the new ROUTER
>>> socket, with a lookup table in the MUX matching the engine-specified
>>> identities to the real routing prefixes, and add extra handshaking to
>>> the connection process to get this information everywhere it needs to be.
>>
>> Yes. It's going to be somewhat more painful, but AFAICS it boils down to:
>>
>> 1. Send a identity message manually after the connect.
>> 2. On the server side, store the identity & connection ID in a map.
>> 3. When request arrives at the server, check the map and forward the
>> message to the right connection.
>>
>> Martin
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>

Jakub Witkowski.