[zeromq-dev] Multicast and logging
zmqdev at a-nugget.de
Fri Apr 16 16:18:21 CEST 2010
Today we've had a discussion on IRC about when one would use IP
multicast and when better not.
The idea was to have a bus-like structure w/ multiple senders and
receivers (PUB/SUB over PGM/EPGM).
Such a structure will become very complicated when using TCP.
And this complexity *will* show up in the performance!
Every publisher has to have a TCP connection with every subscriber.
A lot of connections... all transporting the same data... FAST please!
so the idea was to use IP multicast as a transport.
The use-case for this has been distributed logging, where lots of
machines send their logging messages and an unknown number of consumers
do receive and process them.
I tossed in that most larger sites don't use log-levels anymore and so
you have *lots* of messages flying around. Using IP multicast does help
here because it duplicats network packets only when needed and also
Of course, everything is sent over the network in this model.
This might seem like a waste of resources...
OTOH, consider the following questions: 
1. when do you raise the log-levels?
2. does raising the log-levels change the behaviour of the system?
To shorten the discussino on IRC, I've been asked to provide some
information on that topic.
So here some links from which I gathered my immense knowledge: :-)
How it all began.
If you want so see something working.
You've read that, right? :)
Enjoy! Hope you'll get the picture.
1. When it's to late.
More information about the zeromq-dev