[zeromq-dev] ZMQ_MCAST_LOOP with PGM

Steven McCoy steven.mccoy at miru.hk
Sun Aug 5 01:16:25 CEST 2012

On 2 August 2012 07:20, Pierre Ynard <linkfanel at yahoo.fr> wrote:

> To be clear, if I understand correctly, OpenPGM binds a UDP socket
> to the port given with the multicast address, using SO_REUSEADDR.
> Then it uses this socket to both receive the multicast stream (hence
> SO_REUSEADDR) and the unicast NAKs. This is why if a sender and a
> receiver run on the same host, the socket of only one of them will
> receive unicast NAKs, breaking it for the other one. Right?


So NAKs are sent back upstream with the same destination port as packets
> flowing downstream (the multicasting port). To an outsider like me, this
> feels confusing and counter-intuitive, and looks like a weird quirk.
> I tried to gather the reason from the RFC but I got very confused by
> §8.3: is this really what is described there? Keeping in mind that the
> UDP encapsulation mode is not described by the RFC? Anyway, I don't
> grasp the whole implications of this in the protocol, but it would make
> more sense to me if the NAKs were sent back to the source address and
> port of the sender, back from an address they came from. Also it might
> avoid this problem in the first place.
The protocol is independent of multicast address and IP port.  IBM's UDP
encapsulation makes use of two fixed ports, one for unicast and one for
multicast: this doesn't help the situation at all.

> But more importantly for ZeroMQ; if that's what it is, the problem
> is not really related to multicast loopback, just to running several
> instances on the same host. Then, running several senders (to the same
> group-port) on the same host is a problem too as it breaks reliability
> for all but one of them. And that wasn't solved by removing the
> ZMQ_MCAST_LOOP option.

You have to remember Windows and POSIX platforms have to apply
ZMQ_MCAST_LOOP on different directions:  Windows requires it on the sender,
POSIX requires it on the receiver.

> To be thorough, there would also be some kind of
> security issue on multi-user hosts: a malicious user could open a UDP
> socket on that same port, and would at best DoS the ZeroMQ application
> without it noticing, or at worst spy on leaked information about the PGM
> network topology.

It's reliable multicast: the entire architecture by definition is insecure
and ripe for DoS.  This is not without precedence.

Isn't the problem a bit more complicated than just removing the loopback
> option because it shouldn't be used?

It's been removed because of discussions like this and that 0MQ makes a
host broker trivial to implement: the method other middleware systems
resolve this issue:  TIBCO's Rendezvous Daemon for example.

> This option fits my use case so of
> course I'd welcome its return :) But I'm worried by the rest too because
> regardless of receivers and loopback, I plan on running several senders
> in parallel on the same host.

As long as they are on different ports it is fine, it's an OS multiplexing
issue with a single port.

Look back at the protocol, the options to support multiple senders on a
host are a bit limited:

1)  Only use multicast NAKs.
2)  Include IP source port in the protocol.
3)  Redefine UDP encapsulation for senders to bind to both data-destination
and data-source ports: thus each application on the host has a unique
data-source port to bind to for NAK receipt.

#3 belies the important fact that UDP loopback is terrible and easily
suffers 50%+ packet loss on Linux as normal people use TCP or Unix sockets.
 This is an outstanding issue with the Linux kernel and no one is bothered
enough to fix it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20120804/916ad155/attachment.htm>

More information about the zeromq-dev mailing list