[zeromq-dev] Status on PUB side filtering

[Garrett Smith]

I'm following up on this post from about a year ago:

http://lists.zeromq.org/pipermail/zeromq-dev/2010-July/004705.html

I'm working on migrating away from AMQP to using 0MQ. We make
extensive use of topic exchanges to route message to various
subscribers.

My intention is to build something similar in design to SMTP.
Specifically, messages are sent to recipients using a NAME at ADDRESS
value where ADDRESS is a DNS resolvable host and NAME is some
arbitrary string. Messages would be sent to a PULL socket at ADDRESS
on a known port. The PULL server would forward the message to a PUB
socket for consumption using NAME as the message prefix (or in an
envelope).

My problem with subscriber filtering is its efficiency. Even using
many exchanges, the N x M message traffic between nodes would not be
acceptable.

Because message producers and consumers may live in separate networks
behind firewalls, I can't use a pure peer-to-peer scheme. I'd prefer
to not require any inbound ports to be open -- thus the message
exchange pattern.

At first blush, publish side filtering in 0MQ would address my use
case nicely. I realize the performance and complexity implications of
that model make it a poor (or challenging) fit for the 0MQ socket
layer.

Does anyone have some suggestions for me? Or perhaps I've missed some
important updates on this front.

As an aside, isn't the current PUB/SUB implementation really just
fanout with a trivial filter on the subscriber socket? It's not clear
to me what additional value this form provides. Could it not be called
FANOUT/PULL, relying on subscribers to implement a one line "if
startswith" filter? Or maybe another way of putting this: what is the
use case for subscriber side filtering other than to save that "starts
with" check/discard logic in the application code?

Another aside, I believe Question 1 in:

http://www.zeromq.org/topics:new-topics

should be:

"Should the PUB/SUB topology be simply changed to use PUB side
filtering..."  (currently says "to use SUB side").

My two cents at this time (not yet being as well informed as I should
be): it might make more sense to rename today's PUB to FANOUT and use
a PULL socket to consume from it, dropping the SUB socket altogether.
If/when publish filtering is added, call that PUB. This would require
consumers to filter in the app layer and would obviously break current
SUB code... but maybe that's okay for v3?

Garrett