[zeromq-dev] Load Balancing/Distributing/Queuing Algos: A Discussion

[Stefan Sandberg]

I'd like to suggest something inbetween...

Currently zmq does loadbalancing with the assumption that everything 
connected is of equal quality,
currently just round-robin'izes over that set equally, which only ever 
really makes if you have a rack of identical hardware.

What I'd like is a generic 'choice' device, which will continually rate 
all sockets based on stuff that zmq can
already figure out behind the scenes, like latency/throughput/idletime 
etc etc..

Ie, keep a list of connections, continually try to rate them based on 
their lowlevel behaviour, and then let the
choice-device iterate over a sorted set..
It would involve sorting obviously, but a very well tuned lockless 
priority-queue implementation,
and a customizable update-rate would bring the sorting overhead to a 
minimum, imo.

The default behaviour could simply be the current round-robin, or a more 
fancy randomized set etc,
since they just set weights on each client connection.

Now the nifty bit would be a simple api function to offset the low-level 
socket heuristics already provided by zmq,
with your own custom stuff, like offset a connection by worker cpu/ram etc.

So to summarize, zmq tries to prioritize based on how it's own, 
low-level, sockets are behaving,
but each connection's internal 'worth' can be offset with an api 
function at will.

Just thoughts!
---------------



On 2010-08-17 13:35, Pieter Hintjens wrote:
> On Tue, Aug 17, 2010 at 12:57 PM, Peter Alexander<vel.accel at gmail.com>  wrote:
>
>    
>> Hi Pieter and thanks for your reply to this question in the other thread. I
>> moved it here to see if we could all discuss this some more and come to some
>> possible conclusion and/or road-map.
>>
>> I feel it's important to, in some way, make a selection of
>> balancing/distributing algorithms an optional flag which could maybe part of
>> the flag set in zmq_setsockopts().
>>      
> Discussion is always good.  I see two possible types of distribution algorithm:
>
> * Stateless distribution, which works with no knowledge of the state
> of the workers.  We currently have round-robin but random scatter
> could be an alternative.  This could be trivially added as a socket
> option on REQ/XREQ and PUSH.
>
> * Stateful distribution, which requires data back from workers to make
> an informed choice.  This cannot work with PUSH without significant
> changes.  It can work with REQ/REP if we use the REQ to return state.
> The service (REP side) can then distribute work to the client workers
> (REQ side) based on any algorithm it likes.  I'd assume (but need to
> try) that with XREQ/XREP and identities, it's 100% customizable.
>
>    
>> Using other mechanisms to overcome the built in round-robin, for example,
>> defeats the efficiency of 0mq and therefore it needs to be built-in, imo.
>>      
> Not really... so long as you're not trying to distribute over multiple
> sockets, it's fine.
>
> The other idea is to wait for policy/transport separation
> (http://www.zeromq.org/docs:3_0) and then define your own socket types
> with specific distribution algorithms.
>
> -
> Pieter Hintjens
> iMatix - www.imatix.com
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>