[zeromq-dev] Statistics protocol (ESTP) v0.2

[Paul Colomiets]

Hi Schmurfy, Marten,

On Fri, Jun 8, 2012 at 1:21 PM, Schmurfy <schmurfy at gmail.com> wrote:
> In the intent of building a protocol which is easy to read as a human why
> using obscure symbols to denote the metric "type" ?
> I am talking about <^>,<'> and <+>
>

Yes. I've just realized that too. The original intention was to make
types parametric:

1000'-10:10 -- a derive type with limits from -10 to 10
1234^32bit -- counter with wrap point at 32bit

This way it looks more natural -- as a separator. Then I looked at
collectd and realized that counter that wraps is not used there,
because it's safer to use derive with minimum of 0. And type with
minimum and maximum is coded inside configuration there, so it's hard
to integrate custom limits in collectd, so I've decided to leave this
to the later extensions of the protocol.

Does it make sense now?

> I kinda like the statsd approach on this (https://github.com/etsy/statsd),
> why not using something like this (for the value field):
>
> gauge: 32.45|g
> counter: 129800|c
> derive: 56|d
>

Hm, statd uses semi-readable types. But uses cryptic "|@" to denote interval :)

> always having the type would also be nicer as opposite that having gauge
> considered as default.
>

I think gauge is most simple and understandable type, so it seems
obvious that it's default. See more reasoning below.

> There is also the option to have the type as a proper field:
> ESTP:org.example:sys::cpu: 2012-06-02T09:36:45 10 7.2 g
>

I'm trying to avoid that, because it is possible that some future
revision of the protocol would allow several values in the single
packet (if single value per packet will be a lot of pain in practice).
So I'm trying to keep the protocol extensible to adding more values at
the end. Moving type to the place before the value, might help, but
will look a bit ugly if several values would have different types.

Also parsing a field would encourage to compare whole word:

    if type == "c"

But the current check is expected to be done like the following:

COUNTER_MARKER = "^"
...
    if COUNTER_MARKER in value:

So adding more specific typing would be easier (see possible typing
extension above). Similar reasoning applies to something like
"1234|c", turning it into: "1234|c:32bit" would be a pain.

Similar thing applies to making gauge default. It's nice to have fixed
number of fields, as that makes parsing easier, but if "1234|d" will
be extended to "1234|d|-10|10" to have same field separator for the
words, that doesn't help. And if "1234|d" will be extended to
"1234|d:-10:10" in the future, it doesn't help either, as a lot of
implementations will check "d:-10:10" == "d" so will treat the type as
undefined instead of normal derived type.

How about using just a single latin letter without a separator, and
leave gauge as default?

Any follow up thoughts?

> The resource name is explained but not present in the fields description for
> the example (above), is it the part after the "::" in the application name ?
>

It's between the colons. As description says it's empty for simple
applications/subsystems. Added few more examples. Will try to make
text more clear later.

> Other that these remarks this is starting to look like something I could use
> :)
>

Nice to hear, and thanks for feedback.

> Ps: I sent you a pull request for a typo
>

Thanks. Applied.



-- 
Paul