[zeromq-dev] RE : Proposal for new features in ZMQ.

[Fabien Ninoles]

----- Message d'origine -----
> The preferred way to adding new features is working with master (libzmq 
> repo).

Thanks very much.  The code is quite clean so that should be easy to do.

> > > > > Feature 4 - Add a ready sockopt.

> I guess I don't understand what the option is meant to do. Skip the 
> implementation details, try to explain its usage from user's point of
> view.

The use case is basically to help implementation of pattern like LRU or the COLLECTOR device.  The READY message seems to me a kind of an hack to acknowledge that a new socket has connected to the device, and also force the device to re-implement some internal algo, like lb or dist (not they are complex, but it's just redundant).

However, I just figured out that using the READY flag like I planned to do will not work for LRU if the worker is connected through a proxy like a QUEUE.  So, I need to rethink the pattern;  I could probably just work it with a single label flag, although it would still required the worker to connect to a single device, a limitation not shared by other pattern.

> > > 2. Precisely define the use case you are trying to solve.
> > 
> > Send a request to multiple nodes and collect all replies within a
> > specific time.   That's it.
> 
> Right. That's the 'survey' model discussed lately.
> 
> > > 3. Identify different roles the endpoints can play within the
> pattern.
> > > Create a socket type for each role.
> > 
> > COLLECTOR on sender side, REP on the reply side.   REP doesn't need any
> > special change, just to be able to transport the READY flag.
> 
> Even if it's exactly the same as REP, make it a new socket type, so
> that 
> two patterns are cleanly separated. If in the future you would like
> your 
> "survey respondent" socket to, for example, drop expired surveys you 
> would be able to do it without messing with REQ/REP pattern.

Good advice, although I still find nice if we could expect a worker to connect and work the same whatever the middle device is using for distribution the work load.  But I guess that if we can simply configure a different socket type, it would be quite workable.  Should I try to work on LRU pair ?

> > > 4. Specify exact behaviour of each socket type in terms of routing,
> > > applying backpressure, behaviour in case of failure etc.
> > 
> > COLLECTOR:
> > - Outcoming: Fan-out to READY sockets.
> > - Incoming: Fair-queuing on READY sockets.   Dropped all other
> messages.
> > - Send/Recv: Send, Multiple recv until timeout or all activated.
> 
> Ack.
> 
> > 
> > > 5. Think about scaling. Is interjection of device into the middle of
> > > the topology possible?
> > 
> > The main reason for the introduction of the READY flag is a case of
> > "application scalability".   The current COLLECTOR can only work with a
> > single entry point on all nodes.     Although this can be seen as a good
> > thing (since each request should theorically affect all nodes),   a lot
> > of time I want my request to only be send to a subset of nodes,
> > requiring me to add a "recipients" identifier to the upper protocol,
> > which required the node to be less anonymous that I like (they
> shouldn't
> > care about the topology at all).
> 
> Hm. What's the problem with simply ignoring the surveys you don't want 
> to reply to?

Is more that having a single socket at the top of the chain to handle them all in case I need to seems to be exagerated (and fragile).  If I can allow the frontend of the device to both bind to port and connect to another one, that would help (and even better if I could connect to multiple ports...)
 
Fabien