[zeromq-dev] Zeromq 3.2 pub sub in a large production environment

[Susan Tharakan]

Hi,

I am planning to use zeromq 3.2 pub sub in our production systems and
wanted to check with the community. How stable is zeromq 3.2 for production
use? Has anybody used it in production systems for pub sub model.

For our application the following two requirements are critical

1. Minimize latency
2. Avoid data loss


Currently I'm using a slightly modified version of zeromq 2.0.8 in
production for a 400 machine cluster with reasonable success. The major
issue we've had with zeromq 2.x are

1. High bandwidth utilization during high traffic scenarios because of the
lack of subscriber filtering, possibly resulting in buffer over runs at
producer and drops[data loss].


2. Loosing messages when the subscribers cant keep pace with the producers.
Using the SWAP option in zeromq 2.0 has saved the day in some specific
scenarios as it provided some cushion with  spurty traffic.

With zeromq 3.x the SWAP option seems to be discontinued, which is a big
concern for our usecase, but I'm guessing the buffered queue lengths will
also go down as only required msgs are kept in queue now. Any work arounds
for this problem?

The other important requirement we have is to be able to monitor what's
happening inside of zeromq. We'd done it by writing one of our custom apis
for 2.0 which provided some insight into how much data was buffered at each
end. How easy is it to do this for zeromq 3.x / any plans for it in the zmq
roadmap?

Thanks for your time
Susan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20120718/d805d621/attachment.htm>