[zeromq-dev] How many messages are in queue now?

[Pieter Hintjens]

...

On Fri, Apr 17, 2015 at 5:12 PM, Thomas Rodgers <rodgert at twrodgers.com> wrote:
> Patches welcome.
>
>
> On Friday, April 17, 2015, Ilja Golshtein <ilejncs at narod.ru> wrote:
>>
>> Yes, this is true.
>>
>> If a sysadmin detects that a disk storage is 90% full, she notifies users
>> and do other things to obtain more room, while it is quite possible that the
>> disk is only 89% full at the moment.
>> So, ability to calculate disk usage is "a likely source of confusion and
>> gnashing of teeth".
>> Right.
>>
>>
>> 17.04.2015, 17:06, "Thomas Rodgers" <rodgert at twrodgers.com>:
>>
>> I suppose then it is a question of general utility. Even with ZMQ_PAIR, it
>> is still at best and estimate, likely to have changed (perhaps considerably)
>> from the time you query it to the the time you did something based on that
>> information. Even under the best of circumstances it seems of such limited
>> utility (and a likely source of confusion and gnashing of teeth) that I
>> don't see the value in adding the feature.
>>
>> On Fri, Apr 17, 2015 at 9:00 AM, Ilja Golshtein <ilejncs at narod.ru> wrote:
>>
>> Thomas,
>>
>> Agree that it makes perfect sense for multiple producers and consumers,
>> while in my case it is ZMQ_PAIR.
>>
>> Thanks.
>>
>> 17.04.2015, 16:49, "Thomas Rodgers" <rodgert at twrodgers.com>:
>>
>> At the point a message is placed on the queue it is either at HWM or has
>> at least space for one message, this can be evaluated atomically, claiming
>> the 'slot' under the HWM for the message to be placed on the queue. Any
>> other threads attempting to send would synchronize with this operation and
>> the HWM will be respected, no overshoot.
>>
>> Any observation of an atomic count, as it is being changed, will be the
>> state of the count, at some point in the past. So you could have a HWM of
>> 1000, read a count of 850, make some decision, yah, cool to send, <90%, and
>> have a failing subsequent send, because in the time it took you to do all of
>> that, another thread may have enqueued 30 messages, and yet another thread
>> may have tried to enqueue 50 messages.
>>
>> Any such observations of queue depth with multiple producers and consumers
>> are at best, estimates of queue state at some point in the past, and it is
>> very difficult IME to build reliable logic around that information.
>>
>> On Friday, April 17, 2015, Ilja Golshtein <ilejncs at narod.ru> wrote:
>>
>> Hello Pieter,
>>
>> thank you for your answer.
>>
>> I am in process of reading http://zguide.zeromq.org/hx:chapter7 , while
>> atomic counter seems more natural choice for inproc so far.
>>
>> Could you please explain why it is possible to detect that we are at 100%
>> of HWM and it is not possible to detect that we are e.g. at 90% ?
>>
>> 17.04.2015, 11:54, "Pieter Hintjens" <ph at imatix.com>:
>> > It was never possible, due to the fact that pipes are being written
>> > and read asynchronously. You cannot measure the free space except by
>> > stopping everything.
>> >
>> > The workaround is to use credit based flow control. I've a section in
>> > the Guide that explains how this works. You can in effect know what %
>> > of the pipe from sender to receiver (including all ZeroMQ and network
>> > buffers) is filled.
>> >
>> > -Pieter
>> >
>> > On Wed, Apr 15, 2015 at 4:30 PM, Ilja Golshtein <ilejncs at narod.ru>
>> > wrote:
>> >>  Hello.
>> >>
>> >>  I used to think that it is possible to retrieve number or queued
>> >> messages per socket in recent versions of 0mq, while I failed to find
>> >> correspond getsockopt in 4.0.5
>> >>
>> >>  Use case: I need to alter behavior of my application (drop certain
>> >> type of messages) if a queue is, say, 90% of HWM.
>> >>
>> >>  Please, advise if this facility is implemented or planned, or suggest
>> >> a workaround which is more elegant than having atomic variable (it is
>> >> inproc) maintained by reader and writer threads which is what I plan to do.
>> >>
>> >>  Thanks.
>> >>
>> >>  --
>> >>  Best regards
>> >>  Ilja Golshtein
>> >>  _______________________________________________
>> >>  zeromq-dev mailing list
>> >>  zeromq-dev at lists.zeromq.org
>> >>  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> --
>> Best regards
>> Ilja Golshtein
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> ,
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>> --
>> Best regards
>> Ilja Golshtein
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> ,
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>> --
>> Best regards
>> Ilja Golshtein
>>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>