[zeromq-dev] HWM behaviour & blocking

[Chuck Remes]

On May 9, 2012, at 12:45 AM, Steffen Mueller wrote:

> Dear all,
> 
> I have an application of 0MQ where I have one or multiple server 
> instances that use a PUSH socket to send messages that must be processed 
> by any one of potentially many workers. The servers are single-threaded 
> apart from 0MQ's IO thread and this is hard to change.
> 
> The documentation for the PUSH socket type (and others like XREQ) 
> explain that sending messages on such a socket will be a blocking 
> operation IF
> 
>  - the HWM is hit
>  - OR there are no peers
> 
> In my scenarios, I want to be resilient against intermittent client 
> failure (due to whatever -- coordinated restart, failure, ...). But for 
> this time, the server processes will block on the write and that is not 
> acceptable. Is it reasonable to hope for a way to achieve the following 
> behaviour?
> 
> Sending a messages down a socket of this time will block if:
> 
>  - the HWM is hit for all peers (as per docs, presumably, this means 
> all and each separately since the buffers are probably per-client)
>  - OR a single global HWM is hit if there are no peers
> 
> IOW, I'd like to be able to shove up to $HWM messages down a pipe no 
> matter what, and have the PUSH/whatever socket use a single $HWM-depth 
> buffer if there is no peer. As soon as a peer connects, it could do one 
> of two things:
> 
>  - simply swap that buffer in to become the first connecting peer's 
> send queue (which might be undesirable in some cases since it doesn't 
> load balance but it's likely much easier to implement and more efficient)
>  - use that buffer as another queue stage to load balance from
> 
> Any chance I could have such a functionality? Of course, being able to 
> determine whether there are any peers connected would be great, too.

I recommend that you do non-blocking writes and check the return code. If zmq_errno is equal to EAGAIN, then you know that you have either hit HWM or that there are no peers.

cr