[zeromq-dev] zeromq-dev Digest, Vol 34, Issue 124

[Pieter Hintjens]

On Wed, Oct 27, 2010 at 12:25 PM, Martin Sustrik <sustrik at 250bpm.com> wrote:

> The problem I see is with large scale distributions. In a truly global
> network each node can have potentially millions of peers. It's just not
> viable to monitor presence of them all.

This is a false assumption, IMO.  Even in the largest networks a
normal node never has more than a few hundred or thousand peers, and
scaling is done by intermediation (devices) or simply by clustering.
There are no current examples of networks afaik where nodes have even
1000s of peers, unless they fall into the "big fat server" class.

Typical examples of huge networks where nodes have few or several
dozens of peers: Internet, SIP, bittorrent, human population.

You may find cases of nodes with 10K, 100K, 1M peers but these
_exceptional_ and it is mistaken to equate this with scalability.
It's almost the opposite.  Huge numbers of peers means massive
centralization.  Distribution means nodes with reasonable numbers of
peers and it's totally sane (and necessary) to monitor their presence.

Furthermore _every_ non-trival 0MQ application ends up doing presence
monitoring, heartbeating or somesuch, in patterns where peers are not
100% passive.

-
Pieter Hintjens
iMatix - www.imatix.com