[zeromq-dev] scalability of zeromq in terms fof queues and topics

[Martin Sustrik]

Hi Bhavin,

> I went through every performance test and paper on the zeromq.org site. I
> could not find any benchmark and performance metric that spoke about
> zeromq's performance and scalability from the standpoint of number of topics
> and consumers etc. I have the following queries concerning this - 
>  
> * is there any benchmark or test report that shows how scalability is
> impacted with increase in number of topics (I am assuming since more topics
> may require more processing due to parsing and filtering)

There's no such test. However, the complexity of filtering algorithm 
with respect to total number of topics is O(1).

Complexity w.r.t. number of active subscriptions is O(1).

The only parameter of the algorithm that has O(n) complexity is the 
length of the longest active subscription.

In other words, number of topics and subscriptions is irrelevant to the 
performance. The only relevant parameter is the average length of the 
topic string.

> * is there any upper limit for topics - what should we keep in mind on a
> node w.r.t number of topics

No, there's no limit.

> * same w.r.t subscribers. The performance tests talk about latencies and
> messages per second. How does that change with number of consumer end
> points? Are there any limits / limitations / non-linear scaling issues
> concerning increase in number of consumers connected to a single publisher?

Once again, there are no such tests - the obvious reason is the amount 
of hardware needed.

Anyway, with scenarios where message is delivered to _one_ endpoint 
(REQ/REP for instance) the is O(1), i.e. number of connections is 
irrelevant thanks to O(1) scheduler written by Dhammika Pathirana.

However, if message is delivered to many endpoints (say PUB/SUB) and you 
are using TCP rather than multicast, some work has obviously be done to 
send the message down each connection. Performance results would be 
interesting here...

In case you'll decide to run the tests yourself, please do contribute 
them to the community!

Thanks.
Martin