[zeromq-dev] zeromq low latency requests

[Pieter Hintjens]

Even simpler, check out the task pipeline example in the Guide. You
can run as many task workers on the idle boxes you like, each
connecting to a fixed IP address/port on your main machine. It's not
elegant yet it'll work in a few minutes and all exists in Python.

On Thu, Aug 27, 2015 at 2:42 PM, Joss Gray <joss at jossgray.net> wrote:
> You could use zyre ( https://github.com/zeromq/zyre )  to create a network
> of worker nodes and then distribute tasks and collate results with a central
> server.
>
> On Thu, Aug 27, 2015 at 12:58 PM, Keith Brown <keith6014 at gmail.com> wrote:
>>
>> at the moment I am using Python multiprocessing to process requests
>> but it won't scale well for over 30+ processes since I don't have that
>> many cores. However, I have many desktops in our lab which are idle. I
>> was wondering if its possible to use python multiprocessing module
>> with zero mq to distribute my tasks and get results quickly.
>>
>> BTW, each process takes about 1 second to run but there are thousands
>> I need to run.
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>