[zeromq-dev] Blocking instead of Dropping PUB Socket

[Pieter Hintjens]

Hi Yucheng,

Thanks for the thoughtful explanation and patch. Here are our
contribution policies: http://www.zeromq.org/docs:contributing

So feel free to make a pull request with your change, and add as much
of the explanation as you like into the commit message.

To do this right you'd also want to describe the options on the man
page in docs/zmq_setsockopt.txt.

-Pieter

On Tue, Feb 26, 2013 at 3:37 AM, Yucheng Low <yuchenglow at gmail.com> wrote:
> Hi,
>
> First I would like to thank the developers for a great project. While ZeroMQ
> does not solve *every* communication problem, it is certainly very well
> designed, very effective, and easy to use for the space of problems it can
> solve well. After working with it for a few weeks, some of the reasons for
> certain (what I initially thought unusual) design decisions become clear.
>
> However, I encountered the following PUB/SUB pattern which did not so neatly
> into the patterns described in the guide.
>
> For instance, (just like in the guide), I want to build a replicated key
> value store. However, I want to be able to provide relatively strong replica
> consistency. For simplicity, say there is only 1 master where the contents
> are replicated onto a set of slaves.
>
> This can be implicated easily as a PUB/SUB system, and the Suicidal Snail
> pattern is effective for dealing with slow slaves/subscribers. However, if
> the size of the store is large (many GBs), killing the subscriber, or asking
> for a fresh snapshot, is very costly.  In this case, the right thing to do
> (just as in a database system), will be to put back pressure and slow down
> writes on the master, so that all the slaves can keep up. I therefore would
> like a PUB socket which blocks on hwm instead of dropping messages.
>
> (Alternate designs include using ROUTER on the master and DEALER on the
> slave, and doing manual fanout. However, ROUTER also drops messages on hwm.
> Several alternate designs are also possible by manually designing sequence
> Acks from slaves to master, but that adds substantial update caching
> complexity on the master.)
>
> I therefore made a relatively simple patch as attached to add an additional
> socket option ZMQ_PUB_NODROP  ( and equivalently ZMQ_XPUB_NODROP ). and have
> xsend() in xpub return EAGAIN if not all the pipes matching the message are
> ready for writing.
>
> Initial testing suggests that it seems to work fine, allowing all
> subscribers to receive every message regardless of the hwm. A limitation is
> that I cannot use zmq_poll to test if the PUB socket is writeable. (A
> conservative option is to have the PUB socket be not writeable unless all
> the connected SUB sockets are writeable).
>
> Yucheng
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>