[zeromq-dev] Request-reply broker with one worker and many async tasks

Cosmo Harrigan cosmo.harrigan at singularityu.org
Thu Nov 7 20:26:33 CET 2013


Hi,

After a helpful discussion with Charles, I have created this example of
what I was trying to accomplish:
https://github.com/cosmoharrigan/zguide/blob/master/examples/Python/asyncrrhandlers.py

I am curious to hear feedback on whether this seems like a good approach.

Summary:
Asynchronous request-reply single-threaded server in Python that spawns a
request handler each time a request is received.
This is different from other examples because the number of request handler
threads is not defined ahead of time.

Request:
Client DEALER --> Server ROUTER --> Request handler (spawned)
1. Clients send requests via a DEALER socket on port 5570
2. Server receives requests via a ROUTER socket on port 5570
3. Server passes both the request and the client identity directly to
request handlers when they are spawned

Response:
Client DEALER <-- Server ROUTER <-- Server DEALER <-- Request handler DEALER
1. Request handler returns the response to the Server via a DEALER socket
on inproc
2. Server receives the response from the request handler via a DEALER
socket on inproc
3. Server sends the response to the client via a ROUTER socket on port 5570
4. Client receives the response via a DEALER socket on port 5570

Best regards,
Cosmo Harrigan

On Wed, Nov 6, 2013 at 2:02 PM, Charles Remes <lists at chuckremes.com> wrote:

> I'll paste my response from IRC here since it is relevant and I'll add a
> few more thoughts.
>
> cremes:cosmoharrigan_: how is the response sent back via the callback? If
> it's via that single DEALER socket and the callback runs in another thread,
> you need to protect the socket with a mutex.
> [3:46pm]cremes:Best to just give each thread its own socket. It's a
> cleaner design plus it let's you push thread & lock management into zeromq.
>
> If the reply from the callback needs to be routed back to the client, you
> should continue to use DEALER(client) <-> ROUTER (broker) <-> DEALER
> (worker). In this case there is a single worker but the message passing
> works exactly the same.
>
> cr
>
> On Nov 6, 2013, at 3:43 PM, Cosmo Harrigan <
> cosmo.harrigan at singularityu.org> wrote:
>
> Hi,
>
> I want to implement a design similar to the Request-Reply Broker from the
> zguide (https://raw.github.com/imatix/zguide/master/images/fig17.png) but
> instead of having multiple workers running ahead of time, I want to have
> one worker that receives all the requests, and spawns asynchronous
> operations (tasks) to handle them and send the responses.
>
> What would be the proper combination of socket types to achieve this?
>
> Thanks,
> Cosmo Harrigan
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131107/722c21a4/attachment.html>


More information about the zeromq-dev mailing list