[zeromq-dev] Newbie Bait Request: Debug "Warning" mechanism.

[Pieter Hintjens]

On Tue, Aug 17, 2010 at 7:40 PM, Martin Lucina <mato at kotelna.sk> wrote:

> Two sockets with different defaults? Not pretty :-(

Not pretty.  Makes things more complex.

> I thought some more about just having the default be "" on a SUB
> socket, but that's not nice either since in that case what would a
> subsequent ZMQ_SUBSCRIBE "foo" logically do? "Subscribing to only foo" does
> not seem to fit well with the default.

Longish email follows...

++ Thoughts on the SUB API

This is aimed at a 3.0 design discussion.

If you look at the SUB API in detail, it begins to show design
weaknesses.  It confounds the role of the node (subscriber) with the
search technique (filtering or subscription, it's unclear).  It is
optimized for fast searching on large numbers of prefixes, but it's
doubtful any single subscriber does that.  It doesn't really match the
user's expectations of searching through a data stream by making
increasingly precise requests.  The 'filter' terminology is backwards.
 We have weird edge cases like zombie subscribers.  It feels... wrong.

Redesigning this from the user's perspective would give something like:

* My socket is initially wide open and gets everything (select * from X)
* I can restrict that flow by explicitly setting some criteria (select
* from X where ...)
* I can combine criteria in more and more complex ways (... where A and B or C)

We all agree 0MQ is not SQL.  This is just a universally used pattern
for searching: start broad and narrow it down.  No criteria generally
means "everything".  The SUB API today starts with nothing and then
invites you to broaden it out, because that's how the code works.

In fact it looks possible to make it work more-or-less as the user
expects with no real internal changes.  It's just a matter of language
and presentation, with a few small tweaks.  We can't do complex
operations on criteria because that would break subscription
upstreaming and thus efficient pubsub over TCP. So that's excluded.
But still, this could work:

* My socket is initially wide open and gets everything.
* I can restrict that flow by explicitly setting a prefix (let's use
that term explicitly).
* I can specify further prefixes, which now open the flow carefully
wider (or B or C).
* In the future we might allow negative prefixes (and not D).

If I remove prefixes, that unrolls; when we reach a single prefix
we're back at restricted flow, and when we remove that, we're back at
fully open flow.  Implementation is easy, it's a counter.  The current
matching method is unchanged.

In terms of the API, we:

* Deprecate ZMQ_SUBSCRIBE
* Deprecate the language of 'subscription' and 'filter' in favour of 'prefix'
* Use ZMQ_WHERE to specify a prefix
* Perhaps later add ZMQ_WHERENOT to specify a negative prefix

It is consistent and easy to understand and looks to be easy to read
in code.  It's backwards compatible for almost all cases.  It gives
0MQ that sexy "SQL on steroids" look.  It means we never get zombie
SUB sockets...

Except...! Read on for Return of the Dawn of the Dead...! :-)

++ More Zombies

PULL, XREP, XREQ, and REP sockets can also become zombies, and SUB
sockets as well even given the above change in behaviour.  Here is an
example in Python:

import zmq
context = zmq.Context()
socket = context.socket(zmq.REP)
message = socket.recv()

The application hangs, just like reading off a zombie SUB socket today.

However when I try that with a REQ socket, I instead get an assertion:

zmq._zmq.ZMQError: Operation not supported

I've no real conclusion here except that hanging is useless, and I
like Mato's suggestion of a ENODATA error return.

-Pieter