[zeromq-dev] RE : Proposal for new features in ZMQ.
fabien.ninoles at ubisoft.com
Sun May 15 21:24:16 CEST 2011
----- Message d'origine -----
> The preferred way to adding new features is working with master (libzmq
Thanks very much. The code is quite clean so that should be easy to do.
> > > > > Feature 4 - Add a ready sockopt.
> I guess I don't understand what the option is meant to do. Skip the
> implementation details, try to explain its usage from user's point of
The use case is basically to help implementation of pattern like LRU or the COLLECTOR device. The READY message seems to me a kind of an hack to acknowledge that a new socket has connected to the device, and also force the device to re-implement some internal algo, like lb or dist (not they are complex, but it's just redundant).
However, I just figured out that using the READY flag like I planned to do will not work for LRU if the worker is connected through a proxy like a QUEUE. So, I need to rethink the pattern; I could probably just work it with a single label flag, although it would still required the worker to connect to a single device, a limitation not shared by other pattern.
> > > 2. Precisely define the use case you are trying to solve.
> > Send a request to multiple nodes and collect all replies within a
> > specific time. That's it.
> Right. That's the 'survey' model discussed lately.
> > > 3. Identify different roles the endpoints can play within the
> > > Create a socket type for each role.
> > COLLECTOR on sender side, REP on the reply side. REP doesn't need any
> > special change, just to be able to transport the READY flag.
> Even if it's exactly the same as REP, make it a new socket type, so
> two patterns are cleanly separated. If in the future you would like
> "survey respondent" socket to, for example, drop expired surveys you
> would be able to do it without messing with REQ/REP pattern.
Good advice, although I still find nice if we could expect a worker to connect and work the same whatever the middle device is using for distribution the work load. But I guess that if we can simply configure a different socket type, it would be quite workable. Should I try to work on LRU pair ?
> > > 4. Specify exact behaviour of each socket type in terms of routing,
> > > applying backpressure, behaviour in case of failure etc.
> > COLLECTOR:
> > - Outcoming: Fan-out to READY sockets.
> > - Incoming: Fair-queuing on READY sockets. Dropped all other
> > - Send/Recv: Send, Multiple recv until timeout or all activated.
> > > 5. Think about scaling. Is interjection of device into the middle of
> > > the topology possible?
> > The main reason for the introduction of the READY flag is a case of
> > "application scalability". The current COLLECTOR can only work with a
> > single entry point on all nodes. Although this can be seen as a good
> > thing (since each request should theorically affect all nodes), a lot
> > of time I want my request to only be send to a subset of nodes,
> > requiring me to add a "recipients" identifier to the upper protocol,
> > which required the node to be less anonymous that I like (they
> > care about the topology at all).
> Hm. What's the problem with simply ignoring the surveys you don't want
> to reply to?
Is more that having a single socket at the top of the chain to handle them all in case I need to seems to be exagerated (and fragile). If I can allow the frontend of the device to both bind to port and connect to another one, that would help (and even better if I could connect to multiple ports...)
More information about the zeromq-dev