[zeromq-dev] Notify send()er that they've hit the high water mark

[Paul Colomiets]

Hi Edwin,

On Mon, Jul 9, 2012 at 11:24 PM, Edwin Amsler
<edwinamsler at thinkboxsoftware.com> wrote:
> So here I am, publishing messages through ZeroMQ's send() function at about
> 300MB/s, and my network's set to only send at 10MB/s.
>
> This is kind of a big problem because, while I don't care if the clients
> loose data on their end, the server is either using its memory until it
> crashes, or I'm setting its high water mark and loosing about 29 of every
> thirty messages I produce because I don't know that ZeroMQ can't keep up.
>
> Ideally, when a HWM condition happened, send() would return false, then I'd
> test EAGAIN so I could decide for myself whether I should drop the message,
> or retry later. With that kind of functionality, I could throttle back my
> producer algorithm so that I exactly meet the demand of ZeroMQ instead of
> overwhelming/starving it out.
>
> I'm willing to do the work if this sort of addition makes sense to the rest
> of the project. I'd rather contribute here instead of forking it off in some
> forgotten repository.
>
> Can/should this be done? Is there someone out there willing to mentor me?
>

The behavior is intentional for pub/sub sockets. If you'd have only
one subscriber you could use push/pull. Push sockets block when reach
high water mark, so are Req sockets.

The pub/sub sockets can't reliably block in general case because there
could be multiple subscribers, only one of which reaches high water
mark. Partially this comes from implementation: when you do zmq_send()
zeromq starts to push message to the pipe for each connected
subscribers, if one of the pipes full, there is no way to rollback
messages already put into pipes, if it would it's not clear whether
single slow subscriber should stop the publisher.  Also pgm sockets
can't have backpressure AFAIU.

You can try to implement the behavior as a socket option (it can't be
default behavior), but you have to be aware of the problems above (And
I don't know if core developers are willing to accept the patch).

-- 
Paul