[zeromq-dev] Explicit control over socket buffers?

Charles Remes lists at chuckremes.com
Tue Jun 10 20:14:32 CEST 2014


Maybe it’s time to learn a lesson from nanomsg and eliminate SND/RCV queues altogether. AFAIK, nanomsg pushes all queueing down a layer into the OS and let’s the OS handle this task via whatever mechanisms exist (various buffers, network stack, etc).

cr

On Jun 10, 2014, at 1:09 PM, Pieter Hintjens <ph at imatix.com> wrote:

> It's not just imprecision. The measurement can be several orders of
> magnitude wrong / out of date, particularly when there's a high
> message rate. We know "full" and "empty" and vague mutterings in
> between. Simply measuring the pipe state would interfere with its
> operation. (I believe, happy to be proven wrong).
> 
> I know only two ways to safely manage the number of messages
> explicitly. One is from the client side, using credit-based flow
> control. Second is to offload messages into a single-threaded memory
> queue (a list) that can be managed explicitly by its owning task.
> 
> -Pieter
> 
> On Tue, Jun 10, 2014 at 4:27 PM, Andrew Hume <andrew at research.att.com> wrote:
>> there is a minor way in which i disagree (and have always done so) with
>> this:
>> 
>> there are often potentially many useful ways “internals” can be exposed,
>> even if approximately. for example, there is little reason why an
>> approximate
>> answer to “how many messages are buffered” can’t be generated. and for
>> nearly
>> all cases, knowing the answer +/- a handful of messages would be good
>> enough.
>> and most of the time, allowing a small degree of imprecision allows an
>> inexpensive
>> solution.
>> 
>> of course, YMMV.
>> 
>> andrew
>> 
>> On Jun 9, 2014, at 11:48 PM, Pieter Hintjens <ph at imatix.com> wrote:
>> 
>> It's the same reason TCP doesn't let you discard packets that are on
>> the wire. If you expose the internals of the plumbing to the
>> application then you cannot scale. It makes things considerably more
>> complex, and potentially slower by orders of magnitude. Even trying to
>> expose a concept like "how many messages are queued?" breaks things
>> (libzmq's pipes are filled and emptied asynchronously by two threads
>> and there is simply no way to measure the current state except by
>> freezing everything).
>> 
>> If you want explicit management of message queues, you build this on top.
>> 
>> -Pieter
>> 
>> On Mon, Jun 9, 2014 at 9:24 AM, Dmitry Antipov <antipov at dev.rtsoft.ru>
>> wrote:
>> 
>> Hello,
>> 
>> why 0mq doesn't offer the facilities to control socket buffers (except
>> setting send/receive watermarks and default socket policy to discard/block)?
>> In particular, someone can ask about functions like 1) zmq_peek, to tell
>> how many messages are buffered for input/output, and 2) zmq_discard, to
>> discard all queued messages.
>> 
>> It's quite possible that such a questions just illustrates some
>> misunderstanding
>> about what 0mq sockets are and how they should be used; but can someone
>> explain
>> the (obviously by-design) solution to _not_ provide these features?
>> 
>> Thanks in advance,
>> Dmitry
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
>> 
>> 
>> -----------------------
>> Andrew Hume
>> 949-707-1964 (VO and best)
>> 732-420-0907 (NJ)
>> andrew at research.att.com
>> 
>> 
>> 
>> 
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev




More information about the zeromq-dev mailing list