[zeromq-dev] Migrating IPC-based Pub/Sub System To ZMQ

Pieter Hintjens ph at imatix.com
Mon Aug 16 21:07:43 CEST 2010

On Mon, Aug 16, 2010 at 7:29 PM, Santy, Michael
<Michael.Santy at dynetics.com> wrote:
>> PUSH/PULL (aka DOWNSTREAM/UPSTREAM) will load-balance requests among
>> processors. They are designed precisely for this use case.
> Because of the variability in computation time on the worker nodes we don't
> use round-robin, but instead use least-recently-used (LRU) scheduling.  When
> done processing a worker reports that it is ready, so the scheduling logic
> must be in the application.  If the scheduling was round-robin, the
> DOWNSTREAM/UPSTREAM would be a perfect fit.  It sounds like I'll need to
> come up with some application-level workaround for this use-case.

Indeed, round-robin is not perfect for all types of workload
distribution.  You can consider using a reverse request-reply
mechanism where the workers signal 'ready' by sending a request.

You'd want two threads in the collector.  First thread collects data,
prepares workloads, sends to second thread.  Second thread listens on
REP socket, pops workloads and sends them as replies.

Let me know if you want a worked example, I'm thinking this could make
a nice example for the tutorial of inter-thread signalling.

Pieter Hintjens
iMatix - www.imatix.com

More information about the zeromq-dev mailing list