[zeromq-dev] Proposal for new features in ZMQ.

[Martin Sustrik]

On 05/15/2011 06:35 AM, Pieter Hintjens wrote:
> On Sun, May 15, 2011 at 1:25 AM, Fabien Ninoles
> <fabien.ninoles at ubisoft.com>  wrote:
>
>> Here the feature set:
>> Feature 1 - Add a new msg LABEL flag marking a frame as "labeled".
>
> Part of redesigning the whole message envelope concept, I think. A good idea.

+1

>> Feature 2 - Embedded flags inside the msg_t structure.
>
> Definitely a good idea, it's hard to justify MORE being a socket
> property. In czmq I experimented with this being a message property,
> it works fine.

+1

>> Feature 3 - Add a timeout sockopt.
>
> Nice idea.

+1

>> Feature 4 - Add a ready sockopt.
>
> It'd be impossible IMO to cleanly implement the ready signalling we
> currently do in the LRU pattern because it's tied into application
> processing of data. You could do some tighter coordination between the
> queues at both sides, with low HWM, and this gives you some kind of
> 'ready'. But it won't be on a message-per-message basis afaics.

This is a request for explicit ACKs. Pieter is right that if the ack is 
to be issued only after the message was processed by the app, you are 
going to run into lot of complexity. This way lie distributed 
transactions et al.

>> Feature 5 - The COLLECTOR socket.
>
> Assumes READY can work...
>
>> Most of the current existing pattern would be greatly simplified using
>> them, avoiding weird usage like DEALER socket as worker end point in
>> LRU, and probably unseen and interesting new patterns could probably
>> build using them.
>
> Note that DEALER allows asynchronous worker 'end points', necessary if
> your 'end point' actually deals work out to single threaded worker
> threads. In synchronous workers, we don't use DEALER sockets.

No opinion on this. In any case, if you want to experiment with 
messaging patterns:

1. Define your own messaging pattern. That would give you your own 
vertical segment of the stack which you can experiment with without 
stepping on others' toes.

2. Precisely define the use case you are trying to solve.

3. Identify different roles the endpoints can play within the pattern. 
Create a socket type for each role.

4. Specify exact behaviour of each socket type in terms of routing, 
applying backpressure, behaviour in case of failure etc.

5. Think about scaling. Is interjection of device into the middle of the 
topology possible?

Etc.

Martin