[zeromq-dev] Why ZMQ drop messages?

[Emmanuel TAUREL]

Hi Chuck,

On 25/11/2011 15:50, Chuck Remes wrote:
>>> Hello all,
>>>
>>> I am using ZMQ 3.0.x on linux boxes with  the PUB/SUB pattern.
>>> I have only one subscriber which is very slow. It needs 1 second every
>>> time a message is read.
>>> I have a HWM on the publisher side set to 10.
>>>
>>> In my message, I have a counter which is incremented for each message.
>>> My messages are relatively small (150 bytes)
>>> I have a print of date each time the publisher sends a message
>>> (gettimeofday)
>>> I also have on the same host where the publisher is running a wireshark
>>> tool which captures network packets.
>>>
>>> With wireshark, I see that ZMQ drops messages number 11 to 39. I don't
>>> understand why.
>>> All the previous messages (number 1 to 10) have been sent on the network
>>> because I see them on wireshark
>>> The time reported by wireshark is coherent with the time printed by the
>>> publisher.
> This is exactly how HWM is supposed to work. The "slow subscriber" exerts back-pressure on the publisher. Until the SUB socket responds that it has processed some messages, the PUB socket will drop messages. This is the documented behavior (read the man pages, please).
>
> If you think this behavior is incorrect, please explain how it differs from the documentation.
>
In fact, further tests shows that it does not depends on the time needed 
by the subscriber when it receives a message.
Anyway, I was not clear enough in my explanations.
If I see my messages on the net (wireshark tell me this), this means 
that they are not in the queue anymore.
Messages can't be sent on the wire and in the queue. So, if messages are 
sent on the wire, why ZMQ thinks that the buffer is full and drop 
following messages?

This is to understand this behavior that I have made further studies 
reported in the previous email.

Regards

Emmanuel