[zeromq-dev] Client Server Model in Python

[Brian Granger]

As others have mentioned, you *absolutely* want to run a centralized
queue device that both the client and workers connect to.   This will
load balance the requests to the workers (which will be separate
processes, not threads).  The performance and scalability of this
configuration should be excellent.  The stable release of pyzmq has a
new device function that you can use to start a zmq device:

zmq.device(zmq.QUEUE, socket1, socket2)

Cheers,

Briajn

On Tue, Aug 24, 2010 at 10:39 AM, Oliver Smith <oliver at kfs.org> wrote:
> Oliver Smith said the following on 8/24/2010 12:29 PM:
>
> Abhishek K said the following on 8/24/2010 12:00 PM:
>
> The method I was looking for was
>
> class Extended_thread(Threading.thread):
>         def run(a)
>           #some logic
>
> context=zmq.Context()
> socket=context.socket(zmq.REP)
> socket.bind('tcp://127.0.0.1:5555')
> while True:
>     message=socket.recv()
>            new Extended_thread(socket)
>
> Wait - that's creating a new thread for every message you receive. That's
> going to be incredibly expensive.
>
> What you actually need, apparently, is a zmq_device. I'm not sure how you do
> that in Python.
>
> Again, I think this is another example for the case where it should be
> allowed to bind multiple times to an address over the same context, so that
> multiple threads can service a single port, e.g.
>
> ctx = zmq.Context(1)
> workers = []
>
> def createThreads(numThreads, endpoint):
>     global ctx, workers
>     for i in range(0, numThreads):
>         sock = ctx.socket(zmq.REP)
>         sock.bind(endpoint)
>         thread = new Extended_thread(sock)
>         workers.append(thread)
>
> You can do this with a device, but if you're going to be doing it all over a
> single context, it just seems like so much cpu wastage :)
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com