[zeromq-dev] Load balancing REQ/REP sockets

Martin Sustrik sustrik at 250bpm.com
Thu Mar 18 08:41:15 CET 2010

Hi Brian,

>> The whole XREQ/XREP business is still messy. If you absolutely need it at
>> the moment you can use XREQ to connect and load-balance messages to mutliple
>> REPs. That way you'll get what you need without a need to do any special
>> processing on the REP side.
> I have tried the XREQ -> multiple REP socket topology.  The requests
> are load balanced to the REP sockets and there can be multiple
> requests in the air at a time.  BUT, the notion of load balancing is a
> bit odd.  The load balancing logic is here:
> http://github.com/sustrik/zeromq2/blob/master/src/lb.cpp#L90
> The idea is that the next send will happen on the current+1 active
> pipe.  Basically, this round-robins the requests to the different REP
> sockets.  But, in my mind, this strategy is not a completely useful
> load balancing.  Here is why.  Some requests may take longer/shorter
> for the REP socket to process.  With the current approach, a REP
> socket that handles a very short request, will have to sit there doing
> nothing until its turn comes again.  Likewise, a REP socket that gets
> a very time consuming request will continue to get new requests,
> though it is busy.
> A more useful and dynamic load balancing approach would be would be
> the following:
> * Look at the current + 1 pipe.  If it is not servicing a request, use it.
> * If the current + 1 pipe is busy, see if the current + 2 pipe is
> servicing a request.  If not use it.
> * Continue through the active pipes.  If all are busy, use the current
> + 1 anyways.
> Is there a way of telling if a pipe is servicing a request?  Would it
> make sense to implement this?  This more dynamic load balancing would
> be incredibly useful.

Yes. The meachism intended for this kind of thing is "queue limits" 
mechanism recently ported from 0MQ/1.0.

You can set ZMQ_HWM socket option on a socket, specifying max number of 
messages it can hold. Once the limit is reached for specific connection 
it stops accepting new messages and load balancing skips it in the 

However, the number of requests "on the fly" can be pretty high. There 
can be messages in the sender's queue, in secder's TCP tx window, on the 
network (queue on a switch etc.), in receiver's rx window and in 
receiver's queue.

What you are speaking about is making it work in lock-step fashion, i.e. 
no new message is sent until the reply is received. This should work on 
a REQ socket. If it does not, it should be fixed.

With XREQ socket the situation is a bit more complex. The user can send 
many requests without waiting for a reply. Some of the requests are 
routed through a single connection. At the end of the connection there 
may be another load-balancer, such as zmq_queue. If it worked in 
lock-step fashion. The second load-balancer would be completely useless 
as there would be at most one request processed at a time.


More information about the zeromq-dev mailing list