[zeromq-dev] Design question for multithreaded XREP server

Dr Tune drtune at gmail.com
Sun Dec 19 02:10:25 CET 2010


Hi all,
Firstly,
ZMQ + protocol buffers = awesomeness, thanks! Now I can mix Java and c++ and
it all Just Works.

Secondly;
I have a server which has a single thread listening on an XREP socket and
distributing incoming requests to a pool of worker threads which process
msgs and return a result.
My issue is that b/c I can only use the ZMQ socket in the thread that
created it, the listen/distribute thread must also be the one that returns
the replies, hence I have a loop like this;

1) Poll XREP socket for new messages (non-blocking)
2) If new message, give it to a worker thread (and pass it a handle to a
threadsafe queue we have for replies
3) Check reply queue and send() any on the XREP socket
4) if no messages sent or received, sleep for 1ms
5) loop to 1

The problem being that this loop spins needlessly b/c I can't block on step
1 or step 3; blocking on the XREP means any replies won't get sent until
next request, blocking on 3 can obviously slow things down ridiculously.
I could slow down the spin by increasing sleep in step 4, but that just adds
latency to everything.

Obvious solutions would be;
a) if sockets were threadsafe, pass the socket instead of a queue and allow
each worker thread to write its XREP as soon as it's done. Then you get rid
of steps 3/4 and make step 1 block.
b) run a separate thread for getting queue replies and writing to XREP, this
could block on the queue, but then I'd be using my XREP socket from two
threads.

One workaround I can think of is to disregard the XREQ/XREP thing, use two
threads, open a PULL socket on one thread an a PUSH for replies on the
other. This sucks for a number of reasons not worth going into.

I seem to be solving this wrong. What's the right way to do this?

Many thanks,
DrTune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20101218/1077740d/attachment.htm>


More information about the zeromq-dev mailing list