[zeromq-dev] slow consumer

[Martin Sustrik]

John Skyfox wrote:

> If HWM is set to unlimited the publisher will eventually crash when it 
> will run
> out of resources. What I was thinking (and this is the change that I 
> actually did) is to
> allow the publisher to skip only those consumers that are too slow:
> 
> Before sending a message, the publisher use the function check_write(). 
> if the queue is 
> full (high watermark hit) the publisher will stop writing new messages 
> until the pipe that
> was stalled become available again. On a slow network this can take some 
> seconds.
> 
> What I did is to change the pipe_full() function and the write() 
> function so that when a pipe 
> hit the HWM it will stop writing but just release the message counter. 
> Those consumers that retrieve normally
> will not be penalized because the publisher will still send them new 
> messages as long as they do not hit HWM.
> This is done without any further processing because the pipe_full() is 
> done anyway by each pipe. Since it is done
> during writing, the program run as fast as before with the only 
> advantage that only the slow consumers are missing
> new messages.

Hm. How does that differ from the standard behaviour offered by mainline 
0MQ at the moment? I believe PUB/SUB sockets handle slow consumers 
exactly this way.

Martin