[zeromq-dev] More on ZDCF

[Steffen Mueller]

Hi all,

this is my first post to this list, so hi, I'm Steffen. Quite some time 
ago, I wrote the original Perl bindings for 0MQ, though the current Perl 
module's been rewritten and maintained by Daisuke Maki.

I've always felt that writing applications with many components talking 
to one another using 0MQ was a tad bit annoying since the actual notion 
of who talks to whom tends to get scattered around the code base -- at 
least if written naively. So instead of reinventing 0MQ network 
configuration all over the place, I've written a Perl library that does 
all the configuration and setup based on ZDCF 
(http://rfc.zeromq.org/spec:5). I'm aware that ZDCF defines the 
structure of the configuration data, but not the format in which it is 
written to disk -- if at all. It also validates structural correctness 
of the ZDCF after decoding whatever (pluggable) format it was read in.

The library is called ZMQ::Declare and you can find the early sources at 
https://github.com/tsee/ZMQ-Declare-Perl. There's an examples 
subdirectory with a bunch of simple examples like this:

http://github.com/tsee/ZMQ-Declare-Perl/blob/master/examples/weather/weather_server.pl

Given that ZDCF doesn't even require any textual format at all, I can 
build a normal nested Perl data structure and that'll pass the 
structural validations. Yay. See

https://github.com/tsee/ZMQ-Declare-Perl/blob/master/examples/event_broker/lib/EventBroker.pm#L18

for an example. Furthermore, it's entirely trivial to define a DSL in Perl:

https://github.com/tsee/ZMQ-Declare-Perl/blob/master/author_scripts/dsl_hackery.pl#L103

Though the latter is a tad bit too cute for my taste (even if it's valid 
Perl), so I'd just stick to the actual Perl data structure and/or JSON 
depending on the context.

Before I discovered ZDCF, I had come up with a data structure myself. 
Based on this, I have a couple of comments about the ZDCF spec:

- From much painful experience, I have learned that any format or 
protocol that isn't versioned will come to haunt you one sad night and 
bite you where it hurts the most. Can we please add version information 
to the ZDCF spec? No, the 5 at the end of the URL is not what I am 
talking about. There should be a "version" attribute. See my next point 
about that.

- It's a nested set of objects, "file-global" contains "devices" which 
each contain "sockets". The names of sub-objects share the same 
namespace as fixed properties of the parent object. Eg. you can't have a 
device called "context". While this example isn't a problem at all, it 
means that one is creating backwards incompatibilities when modifying 
the specification in the least by adding another object property since 
it can clash with socket or device names. Maybe it's too late to fix 
this, but having another layer in that a device as "sockets" which is a 
dictionary of socket-name => socket would be swell. (Same for devices.)

- Its scope is too limited. There was a recent thread about somebody 
wishing to use it to configure an entire application that consists of 
many devices which have entirely different threading contexts. I have 
the same desire. Depending on the grade of backwards-incompatibility 
that you are willing to accept (if any. Versions would help here), I can 
try to come up with a data structure to allow for this. Any chances you 
would accept such a change?

- Meta-information. The given format doesn't allow for user-specified 
extra information. Adding an optional, simple string-to-string 
dictionary/map/hash entry for user-specific meta info would be great.


I'd be really happy to get some feedback on both the ZMQ::Declare 
library and the suggestions regarding the ZDCF spec.

Best regards,
Steffen