[zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

[Brian Rossa]

Ian,
I was concerned that this might be the case but -- and these results are highly preliminary -- switching to ipc:// sockets doesn't seem to solve the problem.
Still working on a demonstration.
Cheers!~br



Date: Wed, 11 May 2011 16:27:37 +0100
From: ian.barber at gmail.com
To: zeromq-dev at lists.zeromq.org
Subject: Re: [zeromq-dev] Apparent queuing on PUB/SUB fan-in despite HWM=1

On Wed, May 11, 2011 at 3:46 PM, Brian Rossa <rossabri at hotmail.com> wrote:





My understanding of the HWM semantic is that, when the HWM is reached, new messages will not be queued. That means that the queued messages are not the newest messages. There are good reasons for why this has to be the case. Fine. Still, this growing lag for prolific senders doesn't make any sense to me. The message that I receive at the system endpoint should only be as old as the last zmq_recv from that node. Yet, I am experiencing situations in which one of the intermediate nodes will "take a break" -- eg, it won't send any messages for a while -- and the endpoint will continue to process old messages from it for a few more minutes. Shouldn't it process JUST ONE message from a node that's "taking a break" if HWM is 1 on both sides of that socket pair?


If the messages are reasonably small, this lag could be caused by TCP buffers - your fast node messages are stuck in a big TCP buffer, and ZeroMQ is fair queing from each of the sockets, so takes a little while to consume. The HWM only applies to stuff that has been pulled out of the TCP buffer and into zeromq, so for small messages your effective 'queue' could be a fair bit bigger than 1. 


Ian



_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110511/fb7a7be9/attachment.htm>