[zeromq-dev] What is the best 0MQ socket configuration for an event server?

[Chuck Remes]

On Aug 31, 2010, at 12:57 PM, John McLaughlin - PTIR wrote:

> I have a need that does not seem to have a natural solution in 0MQ, but maybe I am missing something. 
>  
> Here’re the basics:
>  
> There will be a Server process (with a “published” address)
> There will be Client processes that may come and go concurrently from anywhere on the network
> Clients will “subscribe” to certain “events” (the events are client-specific)
> A client’s events will occur over a “long” period of time (minutes or longer)
> Publishing all events to all clients and utilizing client-side filtering is not suitable due the network burden
> I would like a solution than can handle heavy loads of event traffic (1000+ messages/sec) if needed
>  
> One solution I can imagine is if 0MQ supports “dynamically assigned” PUB sockets where the server can request a new PUB socket from 0MQ and let the Port# be auto assigned and query-able.  With this, the subscription reply can carry the new Port# which the client can then “listen to”.


I have worked on something similar using 0mq to what you propose. You need a few more sockets. I recommend using a REQ/XREP pair for client/server. 

When a client activates, it chooses a random port that is not in use and creates a SUB socket listening for all traffic on it. The client then sends a message via REQ to the publisher containing the port that it is listening on. The publisher receives this message, acknowledges it, and creates a new pub socket specific to that client. All published events specific to this client go out that socket.

When the client deactivates, it sends a message to the publisher with the port to deactivate and close.

You end up creating a lot more PUB sockets on your server end and doing all of the filtering at the server. This sounds acceptable to you.

I didn't need to do this to avoid network bandwidth bottlenecks; I created this to enforce some security and entitlements.


I highly recommend that you try out the simpler topology and *verifying* that 0mq cannot keep up with your message rates when publishing all data to all clients. With smart topics the client can reject the data *very* fast. A 1 GB or 10 Gb switched network can also move quite a bit of data without a lot of blocking behavior in the hardware too. You may be able to save yourself a lot of unnecessary work, so just try it.

cr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20100831/74be227a/attachment.htm>