[zeromq-dev] PUB doesn't discard msg when no SUB
sustrik at 250bpm.com
Fri May 27 08:18:09 CEST 2011
On 05/26/2011 10:37 PM, Li, Jia wrote:
> Perhaps a sockopt is the solution here?
> My point is: if the socket, especially PUB/SUB, already decided to drop
> some messages, then perhaps the decision on which message gets dropped
> becomes an application-level decision? No matter which message you drop,
> be it ancient or middle-aged, the truth is some message got dropped -
> that implies the application would need to do some 'recovery' or
> 'sync-up' action (or maybe none at all!), where the application might
> prefer either oldest or newest message.
The application may prefer to drop oldest messages, but my point is that
it's not technically possible.
With a slow consumer the buffers start up filling from receiver side
upstream: first receivers 0MQ rx queue is filled, then receiver's TCP rx
buffer, then sender's TCP tx buffer, then 0MQ's tx queue.
Also note that devices may be interjected into the middle which makes
the list even longer.
Anyway, if you wanted to act when HWM is finally reached on the sender's
side, the only thing you can do is to drop oldest message in sender's
0MQ tx queue. Which won't help much in your scenario. Your receiver app
would still have to receive the load of old messages stored in all the
The altenative approach is to drop oldest messages in each of those
buffers when it fills up. This is what Pieter is proposing AFAICS. One
problem is that you can't do that with TCP buffers. You can do it with
0MQ buffers, but you won't get "drop oldest" semantics. How it would
look like from user's point of view is that random messages would be
missing from the middle of the feed.
More information about the zeromq-dev