[zeromq-dev] push to multiple pull sockets

[Martin Sustrik]

Hi Nathan,

> I asked about this on irc, but I'd like to get more information on this.
> In the system I'm building, I need to push messages to multiple workers.
> The sender knows who all the receivers are, but push sockets only
> support load-balancing and not pushing to all the receivers.
>
> I was told I should use pub/sub for this functionality, but this seems
> needlessly complex if I want to ensure no messages lost as it requires a
> synchronization step. The system I'm building has fault-tolerance built
> at the software layer, so publishers/receivers can change over time as
> machines go down and tasks get reassigned. Synchronizing new publishers
> and receivers mid-processing is not desirable.
>
> Are there technical reasons why multicasting from a push socket to
> multiple pull sockets is not desirable? Alternatively, are there any
> major drawbacks to having the sender open up a separate push socket to
> each receiver?

The messaging pattern for multicasting is PUB/SUB. (PUSH/PULL is 
irrelevant to this discussion).

There are two mutually exclusive options with multicasting (PUB/SUB 
pattern):

1. No messages are ever lost.
2. The distribution of messages never deadlocks.

You can't have both at the same time. If no messages are to be lost, 
single slow/defunct consumer can completely block the whole messagese 
distribution tree.

So, 0MQ opts for 2., no global deadlocks, but messages may be lost.

Martin