[zeromq-dev] Statistics reporting using zeromq/crossroads

[Paul Colomiets]

Hi,

Every time I develop something using zeromq or libxs I come up with a
problem of how to report statistics. Pub/Sub pattern is suited for
statistics reporting very well, but the format of messages is unclear.
The result is ugly and diverse solutions for reporting real-time
statistics.

I think the community would benefit from having common format for
reporting statistics, so that all open-source software could use the
format. And various plugins can be developed for existing statistics
and monitoring systems.


The protocol should meet the following requirements:

1. Simple compact format. Probably textual or fixed-size binary. It
should be easy to craft messages without any dependencies.

2. Each message should bring compact representation of the type
information in itself (so that centralized system doesn't need to be
reconfigured for each new reported value and each software upgrade on
remote system)

3. Subscriber should be able to filter statistics data by pub/sub subscriptions

4. There should be solution to collect that statistics (draw graphs, etc.)


So I have the following questions:

1. What formats are available to (re)use?

2. What protocols and software do you use for your projects?

3. Would you like to participate in standardization of the protocol?



At our company we use collectd. And as zeromq plugin is not in the
mainstream for collectd, and it has complex binary format, we report
using some custom formats and a special application that translates
that into collectd protocol for unix sockets (which in turn nice
textual protocol). Same is done in zerogw (see zerogwstat utility).
This kind of proxying, and the point #2 from the requirements makes me
incomfortable.

--
Paul