[zeromq-dev] Multi-threaded ZeroMQ applications
thijsterlouw at gmail.com
Thu Apr 1 06:28:47 CEST 2010
After discussing the design-problems with a colleague, we came up with this:
1 thread to read and write to the ZeroMQ socket ("zmq-thread")
N threads to do real work ("work-thread")
They communicate via inproc-transport.
The zmq-thread creates an inproc endpoint (bind) and all work-threads
create their own inproc endpoints as well (connecting to the
zmq-thread's inproc). Then in the zmq-thread I use ZMQ_POLL to
zmq_poll() the ZeroMQ-external-socket + Inproc-socket and send/recv.
This way I can send work to the worker-threads and receive responses
...there is one important remaining problem. I want each worker to
connect to several servers; and send the replies back to the same
server. Normally I use XREQ - REP, but now it will break down because
it becomes N(servers) : 1(zmq-thread) : M(work-threads).
Is there any way to make a request stick to the same "session" (so
the reply goes back to the correct server, and not some other server
connected via the same socket) without using REQ-REP sync style? So I
want "async" req-rep.
For example: if I could somehow get access to the session where the
request came from, I could pass this session-id to the worker-thread,
the worker-thread does some work and then returns the result +
session-id and then I could either manually lookup the correct session
to use, or even let ZeroMQ library handle it.
Another solution would be to manually keep a list of sockets inside
the zmq-thread, one socket per server. I would then do the above work
manually, so I know on which socket I need to send the reply. Any
thoughts on this?
On Wed, Mar 31, 2010 at 9:18 PM, Thijs Terlouw <thijsterlouw at gmail.com> wrote:
> Yes exactly, somehow your summary is more clear :)
> On Wed, Mar 31, 2010 at 9:12 PM, Martin Sustrik <sustrik at 250bpm.com> wrote:
>> So what you want is a central application that would lauch requests. The
>> requests are load balanced among several workers. Each worker would in turn
>> load-balance the requests among several worker threads. The replies are to
>> be routed back to the central server. Right?
> Thijs Terlouw,
> Shenzhen, China
More information about the zeromq-dev