[zeromq-dev] Load balancing REQ/REP sockets

[Martin Sustrik]

Hi Brian,

>> The whole XREQ/XREP business is still messy. If you absolutely need it at
>> the moment you can use XREQ to connect and load-balance messages to mutliple
>> REPs. That way you'll get what you need without a need to do any special
>> processing on the REP side.
> 
> I have tried the XREQ -> multiple REP socket topology.  The requests
> are load balanced to the REP sockets and there can be multiple
> requests in the air at a time.  BUT, the notion of load balancing is a
> bit odd.  The load balancing logic is here:
> 
> http://github.com/sustrik/zeromq2/blob/master/src/lb.cpp#L90
> 
> The idea is that the next send will happen on the current+1 active
> pipe.  Basically, this round-robins the requests to the different REP
> sockets.  But, in my mind, this strategy is not a completely useful
> load balancing.  Here is why.  Some requests may take longer/shorter
> for the REP socket to process.  With the current approach, a REP
> socket that handles a very short request, will have to sit there doing
> nothing until its turn comes again.  Likewise, a REP socket that gets
> a very time consuming request will continue to get new requests,
> though it is busy.
> 
> A more useful and dynamic load balancing approach would be would be
> the following:
> 
> * Look at the current + 1 pipe.  If it is not servicing a request, use it.
> * If the current + 1 pipe is busy, see if the current + 2 pipe is
> servicing a request.  If not use it.
> * Continue through the active pipes.  If all are busy, use the current
> + 1 anyways.
> 
> Is there a way of telling if a pipe is servicing a request?  Would it
> make sense to implement this?  This more dynamic load balancing would
> be incredibly useful.

Yes. The meachism intended for this kind of thing is "queue limits" 
mechanism recently ported from 0MQ/1.0.

You can set ZMQ_HWM socket option on a socket, specifying max number of 
messages it can hold. Once the limit is reached for specific connection 
it stops accepting new messages and load balancing skips it in the 
round-robin.

However, the number of requests "on the fly" can be pretty high. There 
can be messages in the sender's queue, in secder's TCP tx window, on the 
network (queue on a switch etc.), in receiver's rx window and in 
receiver's queue.

What you are speaking about is making it work in lock-step fashion, i.e. 
no new message is sent until the reply is received. This should work on 
a REQ socket. If it does not, it should be fixed.

With XREQ socket the situation is a bit more complex. The user can send 
many requests without waiting for a reply. Some of the requests are 
routed through a single connection. At the end of the connection there 
may be another load-balancer, such as zmq_queue. If it worked in 
lock-step fashion. The second load-balancer would be completely useless 
as there would be at most one request processed at a time.

Thoughts?
Martin