[zeromq-dev] Load balancing algorithm

[Brian Granger]

On Thu, Jul 8, 2010 at 5:43 AM, Martin Sustrik <sustrik at 250bpm.com> wrote:
> Brian Granger wrote:
>
> /me back from holiday!

Hope you had a good break!  I am in need of one...

>>> Btw: http://berezecki.com/?p=22
>>
>> Great link, thanks!  This is a brilliant and simple idea as well.
>> Although to implement this, it would require tracking how many
>> messages each client has been sent.  I also liked the shuffle idea
>> that someone commented on below.  I may try these ideas out.
>
> ...
>
>> Yes, but, as Martin brought up, there is also the load balancing
>> effect of the HWM.  When a clients queue fills up to the HWM, the
>> client becomes passive so it won't get any further messages for a
>> while.
>
> We have an estimate of how many messages there are in each pipe.
>
> HWM-style load balancing -- as it is implemented at the moment -- is a
> round-robin over all the pipes that are not full.
>
> Choosing the least filled pipe may be computationally complex (at least
> O(log n)), so using the algorithm described in the article may be a nice
> optimisation.

Is the information about how much each pipe has in it available in the
pipe API.  If so I might give this a shot.

> Still, keep in mind that HWM-style load balancing would work as expected
> only if explicit acks are implemented, so that sender has an estimate of
>  number of messages all the way down to the next node rather than the size
> of its own tx buffer (pipe).

OK.

> Martin
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com