[zeromq-dev] PUB/SUB sockets over PGM intermixes multipart messages from different peers
Martin Sustrik
sustrik at 250bpm.com
Tue Apr 26 16:34:59 CEST 2011
Hi Andrey,
> We're trying to us ZeroMQ to implement N-to-N PUB-SUB scheme where every
> host is both publisher and subscriber. We use PUB/SUB sockets with IPC
> transport to route message inside one host and PGM transport to send
> messages between hosts.
>
> So on each server we have process that creates PUB socket with PGM
> transport and SUB socket on the same PGM endpoint. Local multicast is
> disabled. Actual messages sent are multipart (as PUB/SUB requirement)
> with two parts: tag and body. Everything works completely fine under low
> load: messages published on one host are sent over multicast PGM and
> received on all other hosts. When load goes up (we're observing it at
> around 20 msg/sec) sometimes multipart messages come intermixed with
> each other: instead of two multipart messages [tag1, body1] and [tag2,
> body2] we receive two multipart messages [tag1, tag2, body1] and [body2].
>
> I was digging through ZeroMQ sources (we're on 2.1.4 now) to find out
> the reason for such behavior. In pgm_receiver.cpp every peer (TSI) is
> handled separately, it has its own decoder to do unframing. But received
> messages are sent over the same pipe (inout). My current guess is that
> if two multipart messages are sent over multicast and PGM packets are
> intermixed in such way that first we receive tag1, then tag2, and
> afterwards body1 and body2 (they're coming in that order from decoders),
> ZeroMQ would interpret that as two multipart messages of length 3 and 1
> ([tag1, tag2, body1], [body2]) due to the MORE flags on tag1, tag2
> (being originally first parts of different multipart messages).
Yes. You are right. That's how the problem happens.
> Is this expected behavior? (i.e. ZeroMQ isn't supposed to work in this way?)
Nope. It's a bug. There should be a separate pipe for each TSI. The
functionality in this corner of 0MQ is not fully finished. Would you
like to help with that?
> My only workaround so far is not to use multipart messages, send only
> body and subscribe to '' (drop tags).
You can also define a delimiter of your own to separate topic from body
(say \x00) and thus pass both of them in a single message part. The
subscriptions would work as expected.
Martin
More information about the zeromq-dev
mailing list