[zeromq-dev] Hanging on client "receive" when a responder goes down in extended request-reply pattern (code in C#)

[Chris Whiten]

That was my original thought, and what I had tried before being pointed to
the pattern I originally described.  I've tried setting the high watermark,
but this seems to just dictate how many messages the consumer should buffer
before throwing messages away.  I haven't been able to find any options to
coax the publisher into blocking when this limit is hit.

Here is my push/pull pattern code that I had tried:
http://pastebin.com/CJLaY5zk


On Wed, Jan 15, 2014 at 12:43 PM, Bruno D. Rodrigues <
bruno.rodrigues at litux.org> wrote:

> Dumb question, but why not a simple PUSH/PULL pattern and no queues? Each
> consumer PULLs one message to process. The process pushing data (which may
> be more than one) will block when the HWM is hit (all consumers are busy,
> and the “queue” from the producer is full). No need for “hold-on-i’m-busy”
> messages, as PUSH-PULL does it already by themselves.
>
>
> On Jan 15, 2014, at 16:29, Chris Whiten <chris.whiten at gmail.com> wrote:
>
> I'm using zeromq in a competing consumer setup, where one process is
> pushing data to a set of worker processes to parallelize some data
> processing.
>
> The publisher publishes data much more quickly than the workers can
> process it, so I want the publisher to block when all of the consumers are
> too busy.  I'll decide if they're too busy by having them pull data from
> zeromq and push them to an in-memory queue on each of the workers, and if
> that in-memory queue is too large we temporarily stop accepting messages
> from zeromq.
>
> When all of the consumers are too busy, I wish for the client that is
> pushing data to the consumers to block, so we don't lose large chunks of
> data.  To solve that, I've set up the extended request-reply pattern (as
> outlined in Figure 16 of the zeromq guide at
> http://zguide.zeromq.org/page:all).  This seems to work in the happy
> case, but if a worker goes down the client will never receive a response
> back from that worker, and it will hang indefinitely.  Have I selected the
> wrong pattern, or is there an easy way to remedy this problem?
>
> My code to test this is at http://pastebin.com/CaZpVayu
>
> Thanks for your help
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140115/18bed2ef/attachment.htm>