[zeromq-dev] Asking for a suggestion: creating new connection between clients

[Franz]

Il giorno lun, 05/12/2011 alle 08.05 -0600, AJ Lewis ha scritto:
> On Sat, Dec 03, 2011 at 12:06:16PM +0100, Franz wrote:
> > Hello everyone, I have the following scenario:
> > -1 server, 2 or more clients
> > -the client #1 owns something that interests other client(s), be it a
> > file or a stream of data.
> > -the client #2 does not know the name nor the IP of the client #1, just
> > the one of the server.
> > -Both clients connect to the server, and the server just "makes them
> > able" to connect to each other. It might be something as easy as
> > telling them the IP of the other client, or anything else. The
> > important catch is that the file or stream should not go through the
> > server, but straight from #1 to #2 (for saving bandwidth), more or
> > less the same way it happens in P2P networks.
> > 
> > Any idea about what can be done in ZeroMQ to achieve this?
> 
> Any reason you can't use something that's already built for this, like
> zeroconf for example?  It doesn't use ZeroMQ, but it's used quite a bit
> for service discovery, which is what you're looking for I think.
> 
> Regards,
Ok, the actual problem is something like:
I have to move data around a network of nodes participating to a
computation. There's one host that's dividing the workload, and the
nodes need to talk to each other for sending some data during the
computation. In the end I need something similar to MPI, but with the
possibility of adding nodes "at run time".
The nodes will be put on a LAN (at least in the beginning), but when I
want to add a client, connecting to the server won't be enough: I will
need to connect to peers, EG: I need to know their IP. Sure I can add
the IPs by hand, but I was wondering what can be done to make this as
straightforward as possible, and "safer" (forgetting to add a node to
the list of peers might kill the simulation midway).
Right now I'm using sockets, but ZeroMQ capabilities would be perfect
for my needs (message queues with nice behaviors in heavy load
situation). The only problem is connecting the peers with each others, a
task that is unexpectedly difficult to be automated.