[zeromq-dev] Publisher Side Filtering Revisited

[Martin Sustrik]

Hi Patrick,

> This topic was previously discussed here:
> 
> http://lists.zeromq.org/pipermail/zeromq-dev/2010-June/004194.html
> 
> So I understand why this is done, but it seems to make the 0MQ PUB-SUB 
> topology 
> not very useful for real time market data since it will result in so 
> many unnecessary messages sent.
> This is confusing because the other aspects of 0MQ, low latency and the 
> high rate of small sized messages/sec 
> that can be delivered, seem to be built just for this purpose.

True. Implementing the subscription forwarding is crucial for 
transporting market data over tcp transport. John Dyte have already 
started an implementation effort, however, it's not trivial and any help 
would be appreciated.

> The solution seems to require having symbol and data type subscriptions 
> sent via a PAIR or REQ-REP,
> and then using the PAIR to send data or have have a PUB-SUB with only 
> one SUB.  Either way one 
> is forced implement topologies in ways they were not meant to be used 
> and the value of 0MQ is lessened
> to the point where I have to weigh it against using boost sockets or 
> using something more robust like Apache Qpid.

Exactly. Subscription forwarding is only sane way forward.

> I suppose that I could create hundreds of sockets and partition out the 
> data based on message type and the first letter of the symbol,
> but that would still send a lot of unnecessary messages and would cancel 
> out most of the elegance of 0MQ.

Same as above.

> Anyone know how to approach this problem with 0MQ?

Please, do help with getting the functionality into 0MQ core rather than 
coding workarounds on top of it.

Thanks!
Martin