[zeromq-dev] multiple network interfaces for one exchange?

[Martin Sustrik]

Hi Aamir,

> It's the second use case that I'm interested. We have different
> compute clusters on different networks. We want to combine all of
> these into a single cluster for our use. The only way to do this at
> the 0MQ level would require the ability to define a single global
> object on multiple network interfaces.
> 
> I understand your concerns about not wanting to bloat the API. XML is
> probably a good solution, although I imagine that this would create
> dependencies on third-party libraries such as Xerces-C++.

There's a lightweight XML parsed compiled directly into 0MQ, so this
shouldn't be a problem.

> Would it be
> possible to separate global object creation and pairing the object to
> specific network locations? For example, something like:
> 
> api->create_exchange("E");
> api->add_location("E", "eth0:1234");
> api->add_location("E", "tun0:1234");
> api->remove_location("E", "eth0:1234");
> 
> I'm not sure if this is a good idea or even possible, but it doesn't
> seem to make the API too complicated.

Something like that. My original idea was to simplify the API so that
the only thing application has to do is to create an "endpoint" - i.e.
something you can send messages to or receive messages from (inbound
endpoints can be called queues, outbound endpoint can be called exchanges):

int eid = api->create_exchange ("E");
...
api->send (eid, msg);

All the remaining info will be provided as an XML. something like:

     <endpoint name = "E">
         <bind location = "192.168.0.1:5555" />
         <bind location = "192.168.0.2:5555" />
     </endpoint>

The XML would be preferably stored in a directory service, say:
zmq_server --config-file config.xml)

The nice thing about the solution is that:

1. You have a single point of administration for the whole system rather
than having configuration split over the source code of all the components.

2. With single XML file defining all the dataflows you should be able to
do all kinds of funny stuff: generating nice dataflow graphs, performing
changes to the configuration in automated fashion etc. This would be
particularly useful in enterprise-scale deployments consisting of
hundreds or thousands of components. For improved reliability, the
configuration can be moved to a real directory service like LDAP or AD
later on.

Anyway, that's the long term plan. At the moment we are busy with 
getting 0MQ/0.6 release out of the door, so there's no new functionality 
to be added in next 2-3 weeks. Afterwards, we can think about gradual 
introduction of the concepts described above.

If the thing is needed ASAP we can hack it in for you at consultancy 
rates of course.

Martin