[zeromq-dev] PUB doesn't discard msg when no SUB

[Martin Sustrik]

On 05/26/2011 10:37 PM, Li, Jia wrote:
> Perhaps a sockopt is the solution here?
>
> My point is: if the socket, especially PUB/SUB, already decided to drop
> some messages, then perhaps the decision on which message gets dropped
> becomes an application-level decision? No matter which message you drop,
> be it ancient or middle-aged, the truth is some message got dropped -
> that implies the application would need to do some 'recovery' or
> 'sync-up' action (or maybe none at all!), where the application might
> prefer either oldest or newest message.

The application may prefer to drop oldest messages, but my point is that 
it's not technically possible.

With a slow consumer the buffers start up filling from receiver side 
upstream: first receivers 0MQ rx queue is filled, then receiver's TCP rx 
buffer, then sender's TCP tx buffer, then 0MQ's tx queue.

Also note that devices may be interjected into the middle which makes 
the list even longer.

Anyway, if you wanted to act when HWM is finally reached on the sender's 
side, the only thing you can do is to drop oldest message in sender's 
0MQ tx queue. Which won't help much in your scenario. Your receiver app 
would still have to receive the load of old messages stored in all the 
other buffers.

The altenative approach is to drop oldest messages in each of those 
buffers when it fills up. This is what Pieter is proposing AFAICS. One 
problem is that you can't do that with TCP buffers. You can do it with 
0MQ buffers, but you won't get "drop oldest" semantics. How it would 
look like from user's point of view is that random messages would be 
missing from the middle of the feed.

Martin