[zeromq-dev] Getting rid of low watermark

[Martin Sustrik]

Ok, committed to the trunk.

ZMQ_LWM is still defined so that bindings don't break.

However, attempt to set LWM on a socket would result in EINVAL error.

Martin

Martin Sustrik wrote:
> Hi all,
> 
> Currently, when setting limit for number of messages that can be held by 
>  a socket you have to specify high & low watermarks.
> 
> Here's what documentation says:
> 
> "The ZMQ_HWM option shall set the high water mark for the message queue 
> associated with the socket. The high water mark is a hard limit on the 
> number of outstanding messages in the queue; if this
>        limit has been reached the socket shall enter an "emergency" 
> state and depending on the socket type, 0MQ shall take appropriate 
> action such as blocking or dropping new messages entering the
>        queue."
> 
> "The ZMQ_LWM option shall set the low water mark for the message queue 
> associated with the socket. This option only makes sense when used in 
> conjunction with the ZMQ_HWM option. A socket which
>        has reached it’s high water mark remains in the "emergency" state 
> until the number of outstanding messages in it’s associated message 
> queue falls below the low water mark, at which point normal
>        message processing is resumed."
> 
> As far as I understand -- correct me if I am wrong -- it's not obvious 
> what value LWM should be set to and it's not even clear why the user 
> should bother with setting LWM at all.
> 
> Given the above, I would like to propose an algorithm for automatic 
> setting of LWM.
> 
> Following three points are to be considered:
> 
> 1. LWM has to be less than HWM.
> 
> 2. LWM cannot be set to very low value (such as zero) as after filling 
> the queue it would start to refill only after all the messages are read 
> from it and thus unnecessarily hold the progress back.
> 
> 3. LWM cannot be set to very high value (such as HWM-1) as it would 
> result in lock-step filling of the queue -- if a single message is read 
> from a full queue, writer thread is resumed to write exactly one message 
> to the queue and go back to sleep immediately. This would result in low 
> performance.
> 
> Given the 3) it would be good to keep HWM and LWM as far apart as to 
> reduce the thread switching overhead to almost zero, say HWM-LWM should 
> be 500.
> 
> Still we have to account for the cases where HWM<500 thus driving LWM to 
> negative numbers. My suggestion is to make LWM one half of HWM in such 
> cases.
> 
> The graph showing resulting HWM/LWMs is attached.
> 
> Thoughts?
> Martin
> 
> 
> ------------------------------------------------------------------------
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev