[zeromq-dev] RE : Proposal for new features in ZMQ.

[Martin Sustrik]

Hi Fabien,

>>>> Feature 1 - Add a new msg LABEL flag marking a frame as "labeled".

>>>> Feature 2 - Embedded flags inside the msg_t structure.

>>>> Feature 3 - Add a timeout sockopt.

>
> Nice.  Apart from everything said in the contributing section on the
> site, have you any guidelines on how to approach those first three ?
> Which repository I should clone ?

The preferred way to adding new features is working with master (libzmq 
repo).

1 & 2 are basically the same feature; add the flag to the protocol 
(docs/zmq_tcp.txt); add the flag to the message structure (src/msg.hpp); 
ensure that the label flag is set on envelopes generated by 0MQ itself 
(src/req.cpp, src/xrep.cpp); test properly. Once done we can remove 
stack delimiter messages from the protocol.

3 blocking send/recv are waiting using mailbox_t::recv function. So, you 
have to add timeout there (instead of block parameter); then create new 
socket option (include/zmq.h, src/options.hpp, src/options.cpp); then 
propagate the option value to mailbox_t::recv function.

>>>> Feature 4 - Add a ready sockopt.
>>>
>>> It'd be impossible IMO to cleanly implement the ready signalling we
>>> currently do in the LRU pattern because it's tied into application
>>> processing of data. You could do some tighter coordination between
>>> the queues at both sides, with low HWM, and this gives you some kind
>>> of 'ready'. But it won't be on a message-per-message basis afaics.
>>
>> This is a request for explicit ACKs. Pieter is right that if the ack
>> is to be issued only after the message was processed by the app, you
>> are going to run into lot of complexity. This way lie distributed
>> transactions et al.
>
> I hesitate a lot with this feature because I cannot map it to anything I
> know, neither ACK or something like that.  In fact, it is just the same
> kind of "READY" request of the LRU queue but distribute to all peer
> sockets.
>
> It also seems to me that it is quite simple to implement:
>
> The writer_t pipe is marked active at attachment time. Once a message is
> send to it, the pipe is remove from the active pipes. Once a message
> arrived on the associated reader_t pipe with the READY flag, the
> associated writer_t pipe is mark active again.  Any flaw on this ?

I guess I don't understand what the option is meant to do. Skip the 
implementation details, try to explain its usage from user's point of view.

>> 2. Precisely define the use case you are trying to solve.
>
> Send a request to multiple nodes and collect all replies within a
> specific time.  That's it.

Right. That's the 'survey' model discussed lately.

>> 3. Identify different roles the endpoints can play within the pattern.
>> Create a socket type for each role.
>
> COLLECTOR on sender side, REP on the reply side.  REP doesn't need any
> special change, just to be able to transport the READY flag.

Even if it's exactly the same as REP, make it a new socket type, so that 
two patterns are cleanly separated. If in the future you would like your 
"survey respondent" socket to, for example, drop expired surveys you 
would be able to do it without messing with REQ/REP pattern.

>> 4. Specify exact behaviour of each socket type in terms of routing,
>> applying backpressure, behaviour in case of failure etc.
>
> COLLECTOR:
> - Outcoming: Fan-out to READY sockets.
> - Incoming: Fair-queuing on READY sockets.  Dropped all other messages.
> - Send/Recv: Send, Multiple recv until timeout or all activated.

Ack.

>
>> 5. Think about scaling. Is interjection of device into the middle of
>> the topology possible?
>
> The main reason for the introduction of the READY flag is a case of
> "application scalability".  The current COLLECTOR can only work with a
> single entry point on all nodes.   Although this can be seen as a good
> thing (since each request should theorically affect all nodes),  a lot
> of time I want my request to only be send to a subset of nodes,
> requiring me to add a "recipients" identifier to the upper protocol,
> which required the node to be less anonymous that I like (they shouldn't
> care about the topology at all).

Hm. What's the problem with simply ignoring the surveys you don't want 
to reply to?

> Remarks, the same feature could allow a work to be part of multiple
> LRU queue, threating them in a FQ manner.  This is not always what you
> want but can lead to some interesting new patterns if you can deal with
> this behavior.  Also, since the READY exchange is now part of the socket
> implementation, it can deal with disconnection of transient sockets and
> the LRU could be implement in a simple device.
>
> If you want, I could try to implement an LRU socket, which would be
> simply a DEALER socket with awareness of the READY flag.

Martin