[zeromq-dev] Can a pub-sub endpoint have multiple publishers?

[Randall Nortman]

If you have a small number of nodes/processes, you can implement this
by having every listener know about every sender, or vice versa.  (You
might create a separate service discovery solution).  Then each
listener has a single SUB connected to multiple PUBs; each sender has
a single PUB.  You can either bind the PUBs or bind the SUBs, and the
other side does the connecting.  (I would bind on the side that has
fewer nodes, to make the service discovery problem smaller.)  But as
the number of nodes increases, this becomes unmanageable.  I think it
would work for up to hundreds of nodes, if you implement service
discovery (otherwise tracking nodes and distributing updates will be a
headache).

On Mon, Apr 14, 2014 at 04:38:40PM -0400, Randall Nortman wrote:
> You're correct, my implementation has a single point of failure, which
> is fine in my application.  You could improve upon this by running
> multiple forwarding daemons, but all clients will need to "know about"
> all the daemons so they know where to connect to, where to fail over
> to, how to detect failures, etc.  There are multiple options for a
> setup like this, but nothing that works out of the box as far as I
> know.
> 
> Note also that if you need 100% reliability (that is, you need to know
> that when you send a message it absolutely, definitely reaches all
> listeners), then you are also venturing beyond what ZMQ offers out of
> the box (for good reasons).  Reliable broadcast/fan-out is a difficult
> problem and there's no one-size-fits-all solution.  There are some
> reliable pub-sub patterns in the Guide.  All involve more complexity
> than just PUB and SUB sockets.
> 
> On Mon, Apr 14, 2014 at 03:29:19PM -0500, James Renkel wrote:
> > Randall,
> > 
> > Thank you for your reply.
> > 
> > As I understand your answer, we would have to provide out own daemon.
> > 
> > But that becomes a potential single point of failure, so it would need 
> > to be replicated.
> > 
> > So, don't we then wind up back at the same place and still have the 
> > problem that both multiple publishers and multiple subscribers must each 
> > connect to multiple daemons?
> > 
> > Or am I missing something here?
> > 
> > Jim Renkel
> > Senior Technical Staff
> > mFluent, Inc. LLC
> > mailto:james at mfluent.com
> > 
> > On 14/4/14, 15:05 , Randall Nortman wrote:
> > > I had the same requirement in my application.  You need a forwarding
> > > device (either using the official ZMQ device or your own tiny bit of
> > > code) to make this work.  Yes, you can have multiple PUBs connected to
> > > a single SUB, but there is no way out of the box to connect multiple
> > > PUBs to multiple SUBs.  So you have a daemon that binds both a PUB and
> > > a SUB.  Senders connect a PUB to the daemon's SUB socket, and
> > > receivers connect a SUB to the daemon's PUB, and the daemon just
> > > forwards everything from its SUB to its PUB.
> > >
> > > Instead of a SUB on the daemon, you can just as easily have a PULL
> > > socket, and then the senders have a PUSH.  It's really pretty much the
> > > same thing.  You can even do this with ROUTER/DEALER, but that offers
> > > no advantages.  You must use a PUB socket on the daemon's send side,
> > > because there is no other socket type that does fan-out (deliver one
> > > message to multiple recipients).
> > >
> > 
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>