[zeromq-dev] LRU broker queue in intuitive way on 3.0

[Andrew Hume]

On Apr 6, 2011, at 9:00 AM, Henry Baragar wrote:

> On April 5, 2011 10:30:31 am Andrew Hume wrote:
> > given this degenerate case of a single server and no per-task overhead,
> > then greedy will be your friend. set up one worker thread per cpu,
> > connected to the scheduler by a PULL socket with a queue length of 1.
> > each worker pulls a task description, performs it, and then asks for more.
> > when it receives a 'all done' message, it exits.
> > 
> Basically, that is what I was thinking.  However, my understading is that you have to have all workers connected to the boss before you start sending out tasks. Otherwise the longest running task, as well as the second longest running task get assigned the the first worker that connects (which is very suboptimal).  
> If I have to do a lot of bookkeeping to keep track of the workers, then I suspect piping to subprocesses is an easier solution.  I was intrigued by the BOSS/WORKER pattern email because it sounded like it handled all this bookkeeping, resulting in much simpler code in my application.  
> Now that I have elaborated my use case, is the BOSS/WORK pattern would be appropriate for this use case?


no, you don't have to have all the workers connected.
and if you set the queue length to 1, early arrivees won't snag all teh jobs.

i promise you that using pipes will be more work,
and less flexible to boot.

surrender to the flow, drink the kool-aid, and enjoy the bliss.

------------------
Andrew Hume  (best -> Telework) +1 623-551-2845
andrew at research.att.com  (Work) +1 973-236-2014
AT&T Labs - Research; member of USENIX and LOPSA




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110406/6024e447/attachment.htm>