[zeromq-dev] Identities & PUB/SUB

[Martin Sustrik]

On 04/04/2011 11:32 PM, Paul Colomiets wrote:

>> 2. Publish/subscribe. In this case we assume infinite feed of messages
>> that individual subscribers consume. When shutting down, the subscriber
>> has to cut off the feed at some point.  Whether it cuts off immediately,
>> dropping the messages on the fly or whether it waits for messages on the
>> fly to be delivered is irrelevant. The only difference is that the point
>> of cut off is slightly delayed in the latter case.
>>
> Currently if subscriber disconnected for some reason, it can connect and
> continue where it left over (if its identity is same). But it's useless <snip>

Yuck. That's still in the codebase. I've almost forgot about that. 
Thanks for reminding me!

Here's my proposal for 3.0:

      Disable identities for PUB/SUB pattern.

Rationale:

Identities can be used with PUB/SUB to improve reliability of message 
distribution. However, the messages that are on flight during the 
disconnection/failure are dropped anyway. Moreover, if failure lasts for 
long enough, HWM on publisher is ultimately reached and new messages are 
discarded. Which makes the whole identity thing in PUB/SUB more or less 
useless.

What's more, identities in PUB/SUB hurt robustness of the system as they 
tie resources on the publisher to a dead subscriber. Scenario:

Connect to publisher in loop using generated identities and disconnect 
immediately. In short time, publisher will run out of memory, as it has 
to store messages for those non-existent subscribers.

Even if there's limit for number of concurrent subscribers, the fact 
that "session" remains open even though underlying TCP socket is closed 
means the same scenario would quickly exhaust the session limit and 
create a DoS situation for other users.

Martin