[zeromq-dev] scalability of zeromq in terms fof queues and topics
Martin Sustrik
sustrik at 250bpm.com
Thu Apr 8 15:11:32 CEST 2010
Hi Bhavin,
> I went through every performance test and paper on the zeromq.org site. I
> could not find any benchmark and performance metric that spoke about
> zeromq's performance and scalability from the standpoint of number of topics
> and consumers etc. I have the following queries concerning this -
>
> * is there any benchmark or test report that shows how scalability is
> impacted with increase in number of topics (I am assuming since more topics
> may require more processing due to parsing and filtering)
There's no such test. However, the complexity of filtering algorithm
with respect to total number of topics is O(1).
Complexity w.r.t. number of active subscriptions is O(1).
The only parameter of the algorithm that has O(n) complexity is the
length of the longest active subscription.
In other words, number of topics and subscriptions is irrelevant to the
performance. The only relevant parameter is the average length of the
topic string.
> * is there any upper limit for topics - what should we keep in mind on a
> node w.r.t number of topics
No, there's no limit.
> * same w.r.t subscribers. The performance tests talk about latencies and
> messages per second. How does that change with number of consumer end
> points? Are there any limits / limitations / non-linear scaling issues
> concerning increase in number of consumers connected to a single publisher?
Once again, there are no such tests - the obvious reason is the amount
of hardware needed.
Anyway, with scenarios where message is delivered to _one_ endpoint
(REQ/REP for instance) the is O(1), i.e. number of connections is
irrelevant thanks to O(1) scheduler written by Dhammika Pathirana.
However, if message is delivered to many endpoints (say PUB/SUB) and you
are using TCP rather than multicast, some work has obviously be done to
send the message down each connection. Performance results would be
interesting here...
In case you'll decide to run the tests yourself, please do contribute
them to the community!
Thanks.
Martin
More information about the zeromq-dev
mailing list