[zeromq-dev] push and pull parallel pipeline framework

Keith Brown keith6014 at gmail.com
Sat Oct 3 03:17:32 CEST 2015


I have setup the push-pull with Python
(https://learning-0mq-with-pyzmq.readthedocs.org/en/latest/pyzmq/patterns/pushpull.html)
and it works but I have few questions.

At the moment, the consumer handles it in a synchronous manner. I
would like to have it do it asynchronously. I can't find a good
example to handle multiple requests, or should I be looking at a
different pattern?

If I do it asynchronously, how can I maintain order for the
resultcollector?  I will also have many producers, so do I create a
new context (zmq.PUSH) for each task I want to push to the consumer?

 My goal is to have the producer submit several thousand tasks to the
consumer (there will be 20+ servers farm) which will be load balanced
and I merge the results for the task.

Any help would be great.



More information about the zeromq-dev mailing list