[zeromq-dev] Distributed logging

[Thomas Johnson]

I'm a total 0MQ newbie who is working on writing a distributed logging
framework using 0MQ and Cython. My design goals are that it be fast for the
system generating the logs, and that log messages aren't lost.

The log messages will be consumed by a variety of systems, including an
archiver that writes the logs to disk, as well as a Logstash installation.
Because of the multiple consumers it seems like I want to use PubSub to
send the logs out.

Of course I need to deal with the case where the archiver dies or can't
keep up.

I've read the guide, and my idea is for the publisher to periodically send
REQ messages over a different socket to one "master" archiver asking about
the latest message that the archiver received. If the master doesn't reply
or gets behind, the publisher drops the master, writes to local disk any
log messages not received by the master, and notifies the user of a fault.

Does this sound reasonable or is there a much better way to do this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130819/b2db80a8/attachment.htm>