[zeromq-dev] Subscriptions and HWMs

[Martin Sustrik]

Hi Staffan,

> I'm taking a look at fixing https://zeromq.jira.com/browse/LIBZMQ-299,
> and I'm not quite sure how to tackle it.
>
> There are two problems concerning HWMs and (X)SUB as I see it:
>
> 1. Subscriptions are silently dropped if the SNDHWM is reached. I don't
> know if that's the intended behaviour. If it is, it should probably be
> more clearly documented that subscriptions might be dropped.

It definitely doesn't look like good behaviour. We should decide what 
the right behaviour is though :)

> I know there was talk about making setsockopt(ZMQ_SUBSCRIBE, ...)
> blocking [1], which would solve the "problem" of new subscriptions being
> silently dropped.

Agreed. However, imagine the case where there are multiple publishers 
connected to a single subscriber. In such case we may run into the 
problem of stuck publisher. Ie. if one of the publishers doesn't process 
subscriptions the buffers will ultimately get full and the subscriber 
won't be able to send more subscriptions/unsubscriptions even to other 
well-behaving publishers!

A solution in a short run may be to disconnect the publisher when it 
refuses to process subscriptions (consider it either hung or malevolent).

Solution in the long run is probably separation of "message 
distribution" pattern (one source publishes to multiple destinations) 
and "message aggregation" pattern (many sources publishing to a single 
destination).

> 2. If you have a SNDHWM set on a (X)SUB socket you're quite likely to
> get an assertion at xsub.cpp:218 whenever a pipe is attached or
> hiccuped.

Ack. The cheap and dirty fix to this problem would be to automatically 
set SNDHWM on xsub/sub sockets to infinity (in xsub_t constructor).

> If loosing subscriptions upon reaching the SNDHWM is the expected
> behaviour then 2 is easily fixable by just removing the assertion. If
> not... then I'm not sure how to tackle it.
>
> Bypassing the HWM for subscription messages is one option, but has its
> own caveats.

Right. It can cause process to run out of memory. Still better than the 
assertion probably.

> Keeping track of which subscriptions has been sent is another
> possibility.

What's the idea here?

Martin