[zeromq-dev] Synchronizing Access to a Publisher

etc1 etc1 at nextideapartners.com
Tue May 14 02:12:44 CEST 2013


In our application server, we have multiple threads servicing web requests.
At the end of each request, the thread emits a simple text-based string
indicating the activity performed by the request, like a journal entry. We
want these threads to use ZMQ publishers to send this activity message to
ZMQ subscribers listening on other machines. The QoS we desire is "best
effort" only, some messages could be dropped, but each message must be
published to ALL subscribers available.

 

Single Publisher per App server instance

We thought about initializing a single ZMQ publisher on each app server
instance and synchronizing access to it using locks. But based on the ZMQ
guide, synchronizing thread access to sockets is not a good choice, so we
abandoned this option, not to mention a single instance could easily become
a bottleneck. 

 

Create Publisher On-Demand, Send, then Destroy

In this approach, the executing thread creates a ZMQ publisher on-demand,
publishes the message, then closes the publisher; all other threads do the
same when they need to send a message. While this approach eliminates socket
concurrency issues, it likely performs slower due to the constant buildup
and tear down of publisher instances.

 

Publisher Worker Pool

In this approach, when the app server starts up it creates a pool of ZMQ
publishers, each initialized and ready to publish messages. When an
executing thread wants to publish a message, it requests a publisher from
the worker pool, tells it to send the message, then returns the publisher to
the pool; his approach is identical to a database connection pool. We
synchronize access to the worker pool, but not the publishers since each
thread gets its own temporary publisher when it needs it and returns it when
it no longer needs it.

 

We like the worker pool approach since it's a good balance between
performance and concurrency but without complexity; is this a valid
approach? 

 

Raf

 

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130513/a3e8273c/attachment.htm>


More information about the zeromq-dev mailing list