[zeromq-dev] Hanging on client "receive" when a responder goes down in extended request-reply pattern (code in C#)

[Andrew Hume]

the code seems okay to me (but i don’t use C++), and it is how i have done this in the past
and its worked fine.

the main questions are
1) what is the rate of jobs being produced?
2) what is the relative cost of executing a job?

using PUSH with multiple PULLs works well when jobs are short
(and therefore the normal REQ/REP exchange per job would be too expensive).
use high-water marks to bound how much work each worker has in its queue
(and how much you lose when it goes down). if the cost of a job can be high,
then PUSH/PULL suffers from the problem of one worker filling its buffer
and starving the other workers (not really a problem is the jobs are being
produced at a high rate).

otherwise, each worker should do a REQ/REP (basically saying “give me a job to do”)
to get work.

because i normally like this stuff to be auto configuring, i normally add in a job
queue monitor that automatically fires up extra workers when the job queue gets too long.
that is, a thread that receives jobs (PUSH/PULL) from the producer(s) and serves
REQ/REPs from workers, storing the jobs in a FIFO. this allows accurate counting,
and can support other things like priorities.

On Jan 15, 2014, at 10:00 AM, Chris Whiten <chris.whiten at gmail.com> wrote:

> That was my original thought, and what I had tried before being pointed to the pattern I originally described.  I've tried setting the high watermark, but this seems to just dictate how many messages the consumer should buffer before throwing messages away.  I haven't been able to find any options to coax the publisher into blocking when this limit is hit.
> 
> Here is my push/pull pattern code that I had tried: http://pastebin.com/CJLaY5zk
> 
> 
> On Wed, Jan 15, 2014 at 12:43 PM, Bruno D. Rodrigues <bruno.rodrigues at litux.org> wrote:
> Dumb question, but why not a simple PUSH/PULL pattern and no queues? Each consumer PULLs one message to process. The process pushing data (which may be more than one) will block when the HWM is hit (all consumers are busy, and the “queue” from the producer is full). No need for “hold-on-i’m-busy” messages, as PUSH-PULL does it already by themselves. 
> 
> 
> On Jan 15, 2014, at 16:29, Chris Whiten <chris.whiten at gmail.com> wrote:
> 
>> I'm using zeromq in a competing consumer setup, where one process is pushing data to a set of worker processes to parallelize some data processing.  
>> 
>> The publisher publishes data much more quickly than the workers can process it, so I want the publisher to block when all of the consumers are too busy.  I'll decide if they're too busy by having them pull data from zeromq and push them to an in-memory queue on each of the workers, and if that in-memory queue is too large we temporarily stop accepting messages from zeromq.
>> 
>> When all of the consumers are too busy, I wish for the client that is pushing data to the consumers to block, so we don't lose large chunks of data.  To solve that, I've set up the extended request-reply pattern (as outlined in Figure 16 of the zeromq guide at http://zguide.zeromq.org/page:all).  This seems to work in the happy case, but if a worker goes down the client will never receive a response back from that worker, and it will hang indefinitely.  Have I selected the wrong pattern, or is there an easy way to remedy this problem?
>> 
>> My code to test this is at http://pastebin.com/CaZpVayu
>> 
>> Thanks for your help
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> 
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> 
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev


-----------------------
Andrew Hume
949-707-1964 (VO and best)
732-420-0907 (NJ)
andrew at research.att.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140115/abe75d8e/attachment.htm>