[zeromq-dev] equivalent of accept

[Brian Candler]

> > On Tue, Apr 20, 2010 at 06:45:16PM +0200, Martin Sustrik wrote:
> >> 0MQ does this for you. Actual networking stuff is done in I/O
> >> thread(s). Your application thread is free to do the actual
> >> processing.
> > 
> > Could you also clarify for my benefit: if you have (say) five worker threads
> > which want to handle incoming REQs from clients, should the five threads all
> > call zmq_recv on the same socket, or should the five threads each create
> > their own zmq_socket bound to the same proto/port?
> > 
> > In the first case, it's not clear to me how the zmq_send would route the
> > response back to the correct client - is it implied by which thread is
> > calling zmq_send?
> > 
> > In the second case, a normal socket API would give EADDRINUSE.
> 
> Yes. The right way to do this is:
> 
> Each clinet has a REQ socket to send requests to and receive replies from.
> 
> In the server's main thread create a TCP XREP socket to handle the 
> clients. Create  an inproc XREQ socket to speak to worker threads. Then 
> start the worker threads and run the queue device:
> 
> zmq_device (ZMQ_QUEUE, xrep_socket, xreq_socket)
> 
> The worker threads shoud create an inproc REP socket and connect to the 
> inproc XREQ socket created by the main thread.

Are the semantics of XREQ/XREP sockets documented anywhere? I don't see them
mentioned at http://api.zeromq.org/zmq_socket.html

If I've understood your narrative above, the solution looks like this(*):

    +--------------+
    |        * ------.
    |       REQ    |  |  +----------------------------+
    +--------------+  |  |    zmq_device     workers  |
        Client 1      |  |      *    *   ,--- * REP   |
    +--------------+   `----.  /      \ ,---- * REP   |
    |        * -------------- *        * ---- * REP   |
    |       REQ    |     |   XREP    XREQ             |
    +--------------+     +----------------------------+
        Client 2                    Server
         ... etc

Clearly, a request from client 1 which goes to one of the workers must have
its response routed back to client 1 (and ditto client 2).  Indeed, if
client 1 has multiple TCP connections to the server, the response must be
routed down the *same* TCP connection as the request came in.

So I guess that the XREP and XREQ endpoints must somehow tag each request
to allow the response to be routed back correctly.

If so, are these tags only part of the socket API? Or can you run XREP-XREQ
over TCP, so the tags appear on-the-wire?

> However:
> 
> 1. The XREP/XREQ sockets are broken in a current trunk. I need one full 
> day with no distractions to focus on it :(

Ah, then I can't try it out yet anyway :-)

Regards,

Brian.

(*) Aside: if the REP socket were built into the stack proper I'd expect it
to be thread-safe, so workers could just pop messages directly, without the
zmq_device in the middle. That would simplify the above diagram a lot.

Another aside: it's interesting to note that the workers with a REP socket
open a connection to a REQ socket; that's "backwards" with respect to the
message flow, but a nice way for workers to be added and removed
dynamically.  I've seen something similar done for HTTP, called "swiftiply",
but the project page seems to have gone.