[zeromq-dev] Design questions for an event notification system.

[Renato Samperio]

Hi Joe,

I know this email is quit old but I have not been able to catch up recently with ZMQ mailing list. Did you have any reply related to this email? 

IMHO and experience (in an event notification system) I wouldn't disattend the details of how to implement an XPUB for extensive and fast exchange of messages (e. g. 1 message every 900 ns among more than 1000 members) if you don't manage to have a XSUB/SUB always listening on the other side. As it was in our case. 

Our applications are software components with limitted memory and CPU resources and as far as I know ZMQ does not offer the option to "discard" already queued messages (Hopefully I am wrong in here). So the risk resided on the publishers side, as the memory could grow and crash the application until the subscriber is back on the network. So we ended up, stamping subscribers which are no-matter-what always on and balancing messages among them.

We had also analytically tunned linger times and queue sizes to have an "optimal" performance.

Regards,

Renato.

Date: Thu, 27 Feb 2014 20:59:38 -0800
From: jlin at peaxy.net
To: zeromq-dev at lists.zeromq.org
Subject: [zeromq-dev] Design questions for an event notification system.

Hi, 
I'm planning to implement a notification system that is composed of topics. Event subscribers subscribe to topics and receive events that are published to the topics. The number of topics could be in the range of 5K. Subscribers and publishers are in the range of 10K-50K.

My plan is to use a XPUB-XSUB in a broker to mediate event flow.  I have two options of how to use the broker:1. Use a single broker and have all subscribers connect to it via a SUB socket. A subscriber will make multiple subscribe() calls for the interested topic such as "/music/jazz", "/music/jazz/classic" ...etc.
2. Create separate brokers for each topic and have subscribers connect to multiple brokers. 
Which design will yield better performance? Pros and Cons of each option?

Another question is that assuming I structure the topic as a tree. such as:/ |  ---music      |      --- classic      --- modern

The requirement is that the event subscription should be 'inherited'. e.g. if a subscription is done at '/music/modern', the subscriber would get events published at both '/music', and '/music/modern'. Any suggestion of how this can be done?

Thanks.

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140406/24c96e70/attachment.htm>