[zeromq-dev] Blocking instead of Dropping PUB Socket

Yucheng Low yuchenglow at gmail.com
Tue Feb 26 03:37:23 CET 2013


First I would like to thank the developers for a great project. While
ZeroMQ does not solve *every* communication problem, it is certainly very
well designed, very effective, and easy to use for the space of problems it
can solve well. After working with it for a few weeks, some of the reasons
for certain (what I initially thought unusual) design decisions become

However, I encountered the following PUB/SUB pattern which did not so
neatly into the patterns described in the guide.

For instance, (just like in the guide), I want to build a replicated key
value store. However, I want to be able to provide relatively strong
replica consistency. For simplicity, say there is only 1 master where the
contents are replicated onto a set of slaves.

This can be implicated easily as a PUB/SUB system, and the Suicidal Snail
pattern is effective for dealing with slow slaves/subscribers. However, if
the size of the store is large (many GBs), killing the subscriber, or
asking for a fresh snapshot, is very costly.  In this case, the right thing
to do (just as in a database system), will be to put back pressure and slow
down writes on the master, so that all the slaves can keep up. I therefore
would like a PUB socket which blocks on hwm instead of dropping messages.

(Alternate designs include using ROUTER on the master and DEALER on the
slave, and doing manual fanout. However, ROUTER also drops messages on hwm.
Several alternate designs are also possible by manually designing sequence
Acks from slaves to master, but that adds substantial update caching
complexity on the master.)

I therefore made a relatively simple patch as attached to add an additional
socket option ZMQ_PUB_NODROP  ( and equivalently ZMQ_XPUB_NODROP ). and
have xsend() in xpub return EAGAIN if not all the pipes matching the
message are ready for writing.

Initial testing suggests that it seems to work fine, allowing all
subscribers to receive every message regardless of the hwm. A limitation is
that I cannot use zmq_poll to test if the PUB socket is writeable. (A
conservative option is to have the PUB socket be not writeable unless all
the connected SUB sockets are writeable).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130225/a632b781/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pub_no_drop.diff
Type: application/octet-stream
Size: 6443 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130225/a632b781/attachment.obj>

More information about the zeromq-dev mailing list