[zeromq-dev] Set up a broker as NameServer service
sustrik at 250bpm.com
Thu Nov 26 14:19:47 CET 2009
> First of all thanks for your web site and articles like "broker vs
> brokerless" that is very helpful to understand
> messaging service architectures.
> I am working for a big experiment involving a lot of control command
> among about 500 processes dispatch on more than 100 cpu machines.
> We currently use a simple messaging service called Cm that is working as
> a "broker as NameServer service".
> It was developped 15 years ago and actually does very well its job. It
> makes simple and system independent the task-to-task communication
> problem, by hiding every detail of the use of TCP-IP on which it is
> based. It is peer-to-peer type of communication with a NameServer broker
> daemon that only holds all the connection points associated with a name
> for each process, and allow to monitor the list of processes running,
> and show inter connections, kill them etc.... All processes can
> communicate with any other declared in the same Cm Domain (an
> environment variable), by simply declaring itself with a name to the
> daemon using the CmMessageOpenServer function. To receive messages,
> handler functions are be commonly defined with associated type that
> determine the type of messages to be treated. Very importantly, a
> timeout function CmMessageWaitWithTimeout. can be set not to block the
> running processes waiting for a message Messages can be sent using the
> appropriated type, and a arbitrary serie of strings, double, int, arrays
> data, and the sender can wait to receive the corresponding answer using
> the appropriated handler type response. Message handling is
> asynchronous. Cm is light socket based, but monothreaded, and works only
> on Linux, the Windows platform not being up-to-date. We have also a
> very handy command line executable that allows to send and receive
> answer to any process in the appropriated Cm Domain. So I wonder,
> looking for a new maybe more powerful messaging service, if zeromq could
> be the right one.
Ok, I understand the architecture you are using. It's kind of what 0MQ
is intended for, however, see my comments below...
> What I like is that zeromq is light, multithreaded, multi-platform, and
> seems to be very robust after the tests shown. I think it is a good base
> to develop the type of architecture "broker as a NameServer service"
> But we would need to :
> 1) Write the daemon "broker as a NameServer service" by hiding all the
> IP details
In 0MQ/1.0 there was a zmq_server application that acted as a name
server. However, when you start thinking of how it should work
(especially in faulty environments) you'll find out that what you need
is a distributed, decentralised and replicated directory service.
Writing such an service is major task in itself. Thus with 0MQ/2.0 we've
decided to use DNS to handle the functionality. DNS is already out
there, it's available basically everywhere, it's well designed,
implemented and tested, there are lot of tools, there's support etc.
> 2) Provide the handler function layer, with a simple message writing
> protocol (serie of strings, int, double, arrays)
Here we've followed "do one thing and do it well" maxim. 0MQ does
messaging. To encode the data use any of the specialised libraries
available. Google protocol buffers may be a good choice. You can even
use your exisiting Cm serialisation/deserialisation code.
> 3) Provide a command line executable to send/receive messages
This is related to the previous point. As 0MQ transfers binary BLOBs at
the moment, the only way to design such a command line would be to use a
hex string or a file to define the message content:
./zmq_send message tcp://server001:5555 ff004546474859eeec
It doesn't look very useful IMO.
> I think it would be a very attractive product, for control-command
> oriented environments.
> Looking in the web, I could not find easily any simple product like Cm.
> Have anyone already thought about this type of application in the zeromq
> community ?
I am not aware of such an effort. However, wrapping 0MQ in thin
google-protocol-buffers-based wrapper may be quite a rewarding small
project. Are you interested in writing that kind of thing?
More information about the zeromq-dev