[zeromq-dev] PUB-SUB scalability
steven.p.clark at gmail.com
Mon Aug 16 19:43:59 CEST 2010
On Mon, Aug 16, 2010 at 1:16 PM, Pieter Hintjens <ph at imatix.com> wrote:
> On Mon, Aug 16, 2010 at 4:44 PM, Steven Clark <steven.p.clark at gmail.com>
> > I'm interested in how well PUB-SUB scales via TCP.
> The maths is fairly simple. The network traffic will be sum P(S x M x Z)
> P is the publisher socket
> S is the number of subscribers
> M is the number of messages per second
> Z is the size of messages on average
> Summed up over all publisher sockets.
> > Any guesses as to how high N could go before things go haywire? Is the
> > network bandwidth the limiting factor?
> Yes, that's all.
> > Does the situation improve if I provide a separate PUB socket for each
> > topic?
> Nope. Don't create multiple PUB sockets for this.
Don't create multiple PUB sockets because it's unnecessarily complex, or
because it wouldn't help the network data rate at all? It seems like doing
so *would* reduce data rate / allow more topics/subscribers, especially if
each subscriber A) was only interested in 1 or 2 topics out of a 1000 at a
time, and B) switched topics very infrequently.
If there were many PUB sockets (1 per topic), then when a new client
connects, the marginal increase in network traffic is only that of the 1 or
2 (low rate) PUB sockets going to one more endpoint. Whereas, with only one
PUB socket, a new client connection = ALL data going to one more endpoint.
Maybe PUB-SUB is the wrong pattern for this type of sparse-topic interest,
if one really wants to maximize scalability? (there's also the security
concern of ALL data going to ALL users, which is a whole 'nother bag of
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the zeromq-dev