[zeromq-dev] Pub/Sub over PGM with many topics

[Ambalu, Robert]

Hi, I'm new to zeroMQ and am trying to understand how pub.sub over multicast would work in my scenario.
I want to have one publisher which publishes say 10k different topics.  I can have N subscriber processes, each of which can subscribe to multiple topics
So, use case example.  Say my publisher is publishing market data for 10k different instruments, my subscriber app only handles 100 stocks so it only want to get messages for those 100 stocks.  In addition, I don't want to have my app do any type of topic resolution, but if possible let zeromq handle the topic multiplexing.
So I was thinking on the subscriber side, for every topic I have it looks like I would have to create a separate SUB socket to the same pgm multicast channel, but set with a different SUBSCRIBE topic.  Then I would have to poll on all sockets ( essentially, all topics ) and then dish out to the appropriate pollitem ( which in this case would be my object, so say data for AAPL would be subscribed to my an AAPL pollitem handler which knows what it is / no lookup necessary. ( This is all somewhat trying to mimick what Rendezvous can do )

A few questions here:
- first of all, is this at all proper?
- if I do need to create a socket per topic, and I subscribe to lets say 1000 topics, does that mean I need 1000 sockets, and then for every single poll() iterate through all 1000 sockets? That doesn't seem right / inefficient, so im hoping I'm thinking about this incorrectly
- whats the deal with multipart messages and topics? They seem very intertwined and a bit confusing.  On the publisher side it just sends a multipart message to send a specific topic? But on the receiver side it has to subscribe to said topic in sockopts, and then process the message as a multipart message ( again, I would prefer not to even process the multipart message but just get a regular message for the bound topic )

Hope these questions are clear, thanks in advance

- Rob


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20120525/bb4143ef/attachment.htm>