[zeromq-dev] Subscriptions and HWMs

[Staffan Gimåker]

On Tue, 2012-02-07 at 23:04 +0100, Martin Sustrik wrote:
> Hi Staffan,
> 
> > I'm taking a look at fixing https://zeromq.jira.com/browse/LIBZMQ-299,
> > and I'm not quite sure how to tackle it.
> >
> > There are two problems concerning HWMs and (X)SUB as I see it:
> >
> > 1. Subscriptions are silently dropped if the SNDHWM is reached. I don't
> > know if that's the intended behaviour. If it is, it should probably be
> > more clearly documented that subscriptions might be dropped.
> 
> It definitely doesn't look like good behaviour. We should decide what 
> the right behaviour is though :)
> 
> > I know there was talk about making setsockopt(ZMQ_SUBSCRIBE, ...)
> > blocking [1], which would solve the "problem" of new subscriptions being
> > silently dropped.
> 
> Agreed. However, imagine the case where there are multiple publishers 
> connected to a single subscriber. In such case we may run into the 
> problem of stuck publisher. Ie. if one of the publishers doesn't process 
> subscriptions the buffers will ultimately get full and the subscriber 
> won't be able to send more subscriptions/unsubscriptions even to other 
> well-behaving publishers!

Good point.

> A solution in a short run may be to disconnect the publisher when it 
> refuses to process subscriptions (consider it either hung or malevolent).

But how would you discriminate between stuck and just slow? I think
that'll introduce problems of its own.

> Solution in the long run is probably separation of "message 
> distribution" pattern (one source publishes to multiple destinations) 
> and "message aggregation" pattern (many sources publishing to a single 
> destination).
> 
> > 2. If you have a SNDHWM set on a (X)SUB socket you're quite likely to
> > get an assertion at xsub.cpp:218 whenever a pipe is attached or
> > hiccuped.
> 
> Ack. The cheap and dirty fix to this problem would be to automatically 
> set SNDHWM on xsub/sub sockets to infinity (in xsub_t constructor).

Yep that's what we do.

> > If loosing subscriptions upon reaching the SNDHWM is the expected
> > behaviour then 2 is easily fixable by just removing the assertion. If
> > not... then I'm not sure how to tackle it.
> >
> > Bypassing the HWM for subscription messages is one option, but has its
> > own caveats.
> 
> Right. It can cause process to run out of memory. Still better than the 
> assertion probably.
> 
> > Keeping track of which subscriptions has been sent is another
> > possibility.
> 
> What's the idea here?

If you for each pipe kept track of which subscriptions had been sent you
could try to re-send unsent subscriptions later. It would solve the
problem but it's probably tricky to implement, will cost you memory and
so on.

/S