[zeromq-dev] Load balancing algorithm

[Martin Sustrik]

Brian Granger wrote:

/me back from holiday!

>> Btw: http://berezecki.com/?p=22
> 
> Great link, thanks!  This is a brilliant and simple idea as well.
> Although to implement this, it would require tracking how many
> messages each client has been sent.  I also liked the shuffle idea
> that someone commented on below.  I may try these ideas out.

...

> Yes, but, as Martin brought up, there is also the load balancing
> effect of the HWM.  When a clients queue fills up to the HWM, the
> client becomes passive so it won't get any further messages for a
> while.

We have an estimate of how many messages there are in each pipe.

HWM-style load balancing -- as it is implemented at the moment -- is a 
round-robin over all the pipes that are not full.

Choosing the least filled pipe may be computationally complex (at least 
O(log n)), so using the algorithm described in the article may be a nice 
optimisation.

Still, keep in mind that HWM-style load balancing would work as expected 
only if explicit acks are implemented, so that sender has an estimate of 
  number of messages all the way down to the next node rather than the 
size of its own tx buffer (pipe).

Martin