[zeromq-dev] subscribe/unsubscribe refcounting behaviour

[Pieter Hintjens]

On Wed, Aug 18, 2010 at 12:04 AM, Pieter Hintjens <ph at imatix.com> wrote:

> I've just posted a lengthy proposal for a new SUB API.  I should have,
> but forgot to, cover this particular issue.  Setting prefix criteria
> should be idempotent (i.e. if done again, have no further effect).

Here's that proposal fixed to cover this issue:


++ Thoughts on the SUB API

If you look at the SUB API in detail, it begins to show design
weaknesses.  It confounds the role of the node (subscriber) with the
search technique (filtering or subscription, it's unclear).  It is
optimized for fast searching on large numbers of prefixes, but it's
doubtful any single subscriber does that.  It doesn't really match the
user's expectations of searching through a data set by making
increasingly precise requests.  It doesn't treat identical filters as
the same thing.  The 'filter' terminology is backwards.  Designing
this from the user's perspective would give something like:

* My socket is initially wide open and gets everything (select * from X)
* I can restrict that flow by explicitly setting some criteria (select
* from X where ...)
* I can combine criteria in more and more complex ways (... where A and B or C)

We all agree 0MQ is not SQL.  This is just a universal pattern for
searching: start broad and narrow it down.  No criteria generally
means "everything".  The SUB API today starts with nothing and then
invites you to broaden it out, because that's how the code works.

In fact it's possible to make it work more-or-less as the user expects
with no real internal changes.  It's just a matter of language and
explanation.  We can't do complex operations on criteria because that
would break subscription upstreaming and thus efficient pubsub over
TCP. So that's excluded.  But still, this could work:

* My socket is initially wide open and gets everything.
* I can restrict that flow by explicitly setting a prefix (let's use
that term explicitly).
* I can specify further prefixes, which now open the flow carefully
wider (or B or C).
* In the future we might allow negative prefixes (and not D).

If I remove criteria, that unrolls; when we reach a single prefix
we're back at restricted flow, and when we remove that, we're back at
fully open flow.  Implementation is easy, it's a counter.  The current
matching method is unchanged.

In terms of the API, we:

* Deprecate ZMQ_SUBSCRIBE
* Deprecate the language of 'subscription' and 'filter' in favour of 'prefix'
* Use ZMQ_WHERE to specify a prefix
* Use ZMQ_UNWHERE to remove a prefix
* Perhaps later add ZMQ_WHERENOT to specify a negative prefix
* Make ZMQ_WHERE on a particular prefix /idempotent/

It is consistent and easy to understand and looks to be easy to read
in code.  It's backwards compatible for almost all cases.  It gives
0MQ that sexy "SQL on steroids" look.  It means we don't get zombie
SUB sockets.