[zeromq-dev] pub/sub topic matching on sender side
jon at totient.co.uk
Fri Aug 13 21:52:04 CEST 2010
Martin Sustrik wrote:
Thanks Martin this is most useful.
It's quite complex and I can see the core would need changes.
If initially only the inproc was tackled , is it in anyway simpler, in
> Hi Jon,
> Here's a short summary of what needs to be done to get sender side
> subscription matching.
> First of all it's necessary to establish bi-directional channels for
> pub/sub communication rather than unidirectional as the case is now.
> One direction will be used for passing messages downstream (no change to
> current implemtation) the other will be used to pass subscritptions
> Namely it means providing bidirectional communication in following cases:
> 1. Socket to I/O thread communication (pipes).
> 2. Socket to socket communication (pipes, inproc transport).
> 3. TCP connection
> 4. IPC connection (same as 3)
> Note that multicast (PGM/EPGM) is inherently uni-directional and thus
> cannot be modified this way.
> Each subsription/unsubsctription has to be passed upstream. There are
> several special cases:
> 1. If the next hop is over IPC or TCP, the peer may fail and restart. To
> keep subscriptions working, receiving node has to store all the
> subscriptions and resend them to the peer after reconnection.
> 2. At the points where subscription cannot be passed further up (either
> PUB socket or multicast receiver) there has to be a matching engine,
> filtering the incoming data.
> 3. At the points where we cannot get subscriptions from downstream
> (multicast sender) we have to send all the messages.
> Further, existing matching machine (prefix_tree_t) is a simple filter.
> On a PUB node we have to have one such filter per outbound connection.
> For performance reasons these should be merged into a single message
> matching mechanism.
> Note: With above algorithm, unsubscribe can be delayed. I.e. we
> unsubscribe from particular message feed but still get few messages
> before the unsubscription takes effect. If we believe this is an
> unacceptable behaviour, there has to be a matching engine on each node.
> Next, forwarder device should be modified to pass subscriptions
> upstream. This can be done by creating XPUB and XSUB socket types, new
> socket types that would allow user to explicitly manipulate the
> subscription/unsubscription messages. That way forwarder could receive
> subscriptions from XPUB and send them to XSUB.
> It's not easy, but if you are not frightened and still want to give it a
> try we can discuss what the best first step on the path should be.
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
More information about the zeromq-dev