[zeromq-dev] Survey: use of 0MQ request reply

[Lucas Hope]

Below is only some of the uses I have found for zeromq. I am rolling it out
across most of the software I build now. I also do some simple push-pull and
pub-sub. Amazing product.


> Following some discussion on IRC about reviewing the REQ/REP designs,
> I'd like to measure how people are using these. So, a short survey,
> please answer inline, if you can.
>
> 1. Are you using ROUTER (XREP) sockets in custom devices or brokers?
>

Yes. I have one project based around paranoid pirate with heartbeating, and
one project based around lazy pirate. The lazy pirate requires workers to
talk to a central database which is the main bottleneck, and so I didn't
bother with the heartbeating in that case, since the workers all live on the
same computer as the queue.


> 2. Are you using ROUTER sockets in end-nodes (application code)? If
> so, can you explain how and why?
>

No.


> 3. Are you using DEALER sockets in end-nodes? If so, can you explain
> how and why?
>

Yes. In my PP example above, jobs are requested via REQ dispatch. I also
have a dealer version which manually adds a job-id to the envelope stack.
job results are collated and the next batch of jobs is sent off according to
the results. This is tightly coupled and the requests can come from more
than one source. That is one reason why I chose REQ(dealer)/REP versus
push-pull. Another reason is that the batch only times out when no replies
are coming in.

4. Are you using the REQ-REP multi-hop functionality, either with a
> queue device, or a custom device? Can you explain how and why?
>

I think I explained this in 3. If not, clarify.

5. How do you handle failures (lost requests or replies)?
>

PP above - clients resend lost requests, and heartbeating is used to detect
worker disconnect. Simple reliability and it is working well. The other
(lazy pirate/least recently used) version is a remote procedure call
transport, and so the clients will have different characteristics -
application specific.

6. Anything you feel is missing or inadequate in the REQ/REP designs?
> Please explain with a use case.
>

It would be useful to have a socket option to drop REQ (or DEALER) requests
from a REP (or ROUTER) when the socket has been deemed to be unresponsive -
when it is known that it won't receive a reply.

In the paranoid pirate example: say for some reason all workers die, but the
queue remains active.  The client gives up on a great many jobs, and
resends. It may do this multiple times before abandoning.  These abandoned
jobs fill the queue and when the workers reconnect, they will start on the
abandoned jobs first, which can stop them from working on active tasks until
it is too late because client has already given up. Thus the lack of this
option turns a recoverable situation into a failure.

(A work-around where the client sends an "ABORT" message before aborting is
possible, but that doesn't work for REQ sockets due to the strict
send-receive pairing.)

Peace, and thanks.

-Luke
-- 
---------------------------------------------------
Dr Lucas Hope - lucas.r.hope at skype
Machine Learning and Software Engineering Consultant
Melbourne, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110616/ef95d55c/attachment.htm>