[zeromq-dev] Draft proposal for revamped ZMQ_MONITOR functionality

Lourens Naudé lourens at methodmissing.com
Tue Oct 1 16:39:03 CEST 2013


+1


On Tue, Oct 1, 2013 at 3:32 PM, Pieter Hintjens <ph at imatix.com> wrote:

> Some thoughts on this...
>
> - I really like the idea of text events that can be easily parsed, and
> trivially printed for debugging
> - Instead of socket numbers, we could use printable identities
> - Instead of event numbers, we could use short string constants
> - name=value pairs is a great idea
>
> -Pieter
>
>
> On Tue, Oct 1, 2013 at 4:09 PM, gonzalo diethelm <gdiethelm at dcv.cl> wrote:
>
>>  OK, this is what I am thinking, in a very rough first cut, of how
>> ZMQ_MONITOR should work. Some notes follow. Comments are welcome.****
>>
>> ** **
>>
>> **1.       **This proposal entails a change in the monitor API.****
>>
>> **2.       **The purpose is to provide more event types and more room
>> for growth, and ease support of this functionality from bindings, where
>> dealing with binary data could be complex.****
>>
>> **3.       **There is an assumption there won’t be much data transmitted
>> with this mechanism, hence I have favored clarity and simplicity over
>> efficiency.****
>>
>> **4.       **I tried to cover all possible event types likely to be of
>> interest. There may be more. There may be better names for the event types.
>> The proposed event types may already be covered in the current
>> implementation.****
>>
>> **5.       **Perhaps EVENT and SOCKET (see below) should be 64 bit
>> quantities?****
>>
>> **6.       **Perhaps EVENT should be on a separate frame, as it is today?
>> ****
>>
>> **7.       **I am not sure how to denote a (local or remote) endpoint.
>> Should it be a char* (“tcp://192.168.11.5:5678”) or a 32 (64) bit value
>> (file descriptor, pointer)?****
>>
>> **8.       **To make it even easier, each line could have the format
>> “key=value”. That would make it a breeze in some languages to read all
>> lines into a dictionary data structure.****
>>
>> ** **
>>
>> Usage: as before, after calling zmq_socket_monitor() for certain sockets,
>> on a different thread you would create a ZMQ_PAIR socket, connect that
>> socket to the inproc endpoint specified with zmq_socket_monitor() and read
>> messages in a loop.  Al messages are strings of the form:****
>>
>> ** **
>>
>>   EVENT****
>>
>>   SOCKET****
>>
>>   [additional lines]****
>>
>> ** **
>>
>> EVENT is an 8 byte, base 16, 32 bit numeric value identifying the event
>> type that was triggered. Example: “45a2887f”.****
>>
>> SOCKET is an 8 byte, base 16, 32 bit numeric value identifying the local
>> socket where the event was triggered. Example: “8721b4ce”.****
>>
>> ** **
>>
>> After these two fields there may be zero or more additional lines, with
>> one field per line; each possible event type has its own predefined set of
>> fields.****
>>
>> ** **
>>
>> Supported event types (and their additional fields) follow.  Those marked
>> with (*) require a better, more concise name.****
>>
>> ** **
>>
>> ZMQ_EVENT_BIND_COMPLETED: current local socket bind on local endpoint
>> completed****
>>
>>   local endpoint****
>>
>> ** **
>>
>> ZMQ_EVENT_UNBIND_COMPLETED: current local socket unbind from local
>> endpoint completed****
>>
>>   local endpoint****
>>
>> ** **
>>
>> ZMQ_EVENT_CONNECT_COMPLETED: current local socket connection to remote
>> endpoint completed****
>>
>>   remote endpoint****
>>
>> ** **
>>
>> ZMQ_EVENT_CONNECT_DELAYED: current local socket connection to remote
>> endpoint delayed****
>>
>>   remote endpoint****
>>
>> ** **
>>
>> ZMQ_EVENT_CONNECT_RETRIED: current local socket connection to remote
>> endpoint retried****
>>
>>   remote endpoint****
>>
>> ** **
>>
>> ZMQ_EVENT_ACCEPT_COMPLETED: remote endpoint connection to current local
>> socket accepted****
>>
>>   remote endpoint****
>>
>> ** **
>>
>> ZMQ_EVENT_ACCEPT_FAILED: remote endpoint connection to current local
>> socket failed****
>>
>>   remote endpoint****
>>
>> ** **
>>
>> (*) ZMQ_EVENT_LOCAL_DISCONNECT_COMPLETED: current local socket
>> disconnected from remote endpoint****
>>
>>   remote endpoint****
>>
>> ** **
>>
>> (*) ZMQ_EVENT_REMOTE_DISCONNECT_COMPLETED: remote endpoint disconnected
>> from current local socket****
>>
>>   remote endpoint****
>>
>> ** **
>>
>> ZMQ_EVENT_CLOSE_COMPLETED: current local socket close completed (useful
>> for tracking progress in term())****
>>
>>   NONE****
>>
>> ** **
>>
>> ZMQ_EVENT_CLOSE_FAILED: current local socket close failed (useful for
>> tracking progress in term())****
>>
>>   NONE****
>>
>> ** **
>>
>> ZMQ_EVENT_CORRUPTED: current local socket corruption detected. When does
>> this happen?****
>>
>>   NONE****
>>
>> ** **
>>
>> ZMQ_EVENT_LINGERED: current local socket linger period expired****
>>
>>   linger period. Example: “1000” => 1 second****
>>
>> ** **
>>
>> (*) ZMQ_EVENT_REMOTE_IDENTIFIED: current local socket ascertained remote
>> endpoint's identity****
>>
>>   remote endpoint****
>>
>>   remote identity (binary, 1 to 255 bytes, not starting with 0). Example:
>> “68656c6c6f” => “hello”****
>>
>> ** **
>>
>> ZMQ_EVENT_SNDHWM_REACHED: send high water mark reached****
>>
>>   value of SNDHWM (number of messages). Example: “100” => 100 messages.**
>> **
>>
>> ** **
>>
>> ZMQ_EVENT_RCVHWM_REACHED: receive high water mark reached****
>>
>>   value of RCVHWM (number of messages). Example: “200” => 200 messages.**
>> **
>>
>> ** **
>>
>> ** **
>>
>> Maybe add events for these conditions as well:****
>>
>> subscription registration received from downstream****
>>
>> subscription unregistration received from downstream****
>>
>> ** **
>>
>> -- ****
>>
>> Gonzalo Diethelm****
>>
>> DCV Chile****
>>
>> ** **
>>
>> *From:* zeromq-dev-bounces at lists.zeromq.org [mailto:
>> zeromq-dev-bounces at lists.zeromq.org] *On Behalf Of *
>> Richard_Newton at waters.com
>> *Sent:* Saturday, September 28, 2013 7:03 AM
>> *To:* ZeroMQ development list
>> *Subject:* Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor****
>>
>> ** **
>>
>> Formalizing the protocol would be good, but a default endpoint would make
>> it difficult to monitor 2 different sockets from 2 different places in code
>> would it not?  Not sure if anyone uses this use case but I could imagine a
>> large zeromq based application might want to.
>>
>> Ric.
>>
>>
>> [image: Inactive hide details for "Pieter Hintjens" ---28/09/2013
>> 09:50:33 AM---These sound like good ideas. I think we lack documentat]"Pieter
>> Hintjens" ---28/09/2013 09:50:33 AM---These sound like good ideas. I
>> think we lack documentation on the event interface, and it could be w
>>
>> From: "Pieter Hintjens" <ph at imatix.com>
>> To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org>,
>> Date: 28/09/2013 09:50 AM
>> Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
>> Sent by: zeromq-dev-bounces at lists.zeromq.org****
>>  ------------------------------
>>
>>
>>
>>
>> These sound like good ideas. I think we lack documentation on the
>> event interface, and it could be worth wrapping in CZMQ too.
>>
>> I think formalizing the internal protocol would be good, and using a
>> default inproc endpoint as we do with ZAP could make it simpler.
>>
>> On Fri, Sep 27, 2013 at 8:57 PM, gonzalo diethelm <gdiethelm at dcv.cl>
>> wrote:
>> > I have been looking into the monitor mechanism and I think there are
>> several
>> > shortcomings, that I would like to address. I would like some opinions
>> on
>> > these points before diving into the code.
>> >
>> >
>> >
>> > 1.       We really need two separate “disconnected” events: one
>> > “local_close” for when the local socket has disconnected from its remote
>> > peers, another “remote_disconnected” for when one (out of possibly
>> many) of
>> > the socket’s remote peers has disconnected from us. In the second case,
>> it
>> > is necessary to identify the remote peer (by its address?).
>> >
>> > 2.       We should also add a new type of event (as Doron Somech
>> suggested
>> > half a year ago) that will trigger whenever the peer’s identity is
>> > ascertained by the local socket. This would allow us to then identify
>> which
>> > was the peer that produced a “remote_disconnected” event.
>> >
>> > 3.       I think (but have not thought this through) it would be easier
>> for
>> > bindings if all the event data (except perhaps the event id) was
>> formatted
>> > as ASCII strings, otherwise it is necessary to interpret binary data in
>> > other languages.
>> >
>> > 4.       I think we should switch from 16 to 32 bits for the event
>> mask, we
>> > have already used up the first 10 bits.
>> >
>> >
>> >
>> > Best regards,
>> >
>> >
>> >
>> > --
>> >
>> > Gonzalo Diethelm
>> >
>> > DCV Chile
>> >
>> >
>> >
>> > From: zeromq-dev-bounces at lists.zeromq.org
>> > [mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>]
>> On Behalf Of Joel Lauener
>> > Sent: Wednesday, August 07, 2013 9:53 AM
>> > To: zeromq-dev at lists.zeromq.org
>> >
>> >
>> > Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
>> >
>> >
>> >
>> > Hi Richard,
>> >
>> >
>> >
>> > Thanks for your answer. I already receive the address but in the
>> particular
>> > case of a router socket it's not that useful. I receive something like
>> > 'tcp://0.0.0.0:2000' which is the address to which the socket is bound.
>> >
>> >
>> >
>> > My question was if there is a way to know what is the identity of the
>> peer
>> > that has been disconnected (as set on the client side with the
>> ZMQ_IDENTITY
>> > option). There are many connections behind a router. When I receive a
>> > disconnect event I need to know to which connection it belongs in order
>> to
>> > take the correct action on the server (callback user code, cleanup
>> resource,
>> > ..).
>> >
>> >
>> >
>> > Cheers,
>> >
>> > Joel.
>> >
>> >
>> >
>> > Date: Mon, 5 Aug 2013 11:12:34 +0100
>> > From: Richard_Newton at waters.com
>> > Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
>> > To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org>
>> > Cc: zeromq-dev-bounces at lists.zeromq.org
>> > Message-ID:
>> >
>> > <OF242DDFB2.16EF4ADC-ON80257BBE.0037B5C6-80257BBE.00381560 at waters.com>
>> > Content-Type: text/plain; charset="us-ascii"
>> >
>> >
>> > Hi Joel,
>> >
>> > The monitor event mechanism uses 2 sends to communicate, the first
>> message
>> > contains the event details and fd, the second will contain the address,
>> so
>> > you need to do a second receive to get the address.
>> >
>> > Apparently this was to make it easier for language bindings.
>> >
>> > Ric.
>> >
>> >
>> >
>> >
>> > From:   "Joel Lauener" <Joel.Lauener at cern.ch>
>> > To:     "zeromq-dev at lists.zeromq.org" <zeromq-dev at lists.zeromq.org>
>> > Date:   05/08/2013 11:02 AM
>> > Subject:        [zeromq-dev] ROUTER socket and zmq_socket_monitor
>> > Sent by:        zeromq-dev-bounces at lists.zeromq.org
>> >
>> >
>> >
>> > Hi,
>> >
>> > Currently I'm trying to setup a monitor on a ROUTER socket to detect
>> clients
>> > disconnections. This system is put in place in addition to an heartbeat
>> > mechanism in order to provide faster disconnection detection on the
>> server
>> > side.
>> >
>> > My problem now is that I effectively receive a ZMQ_EVENT_DISCONNECTED
>> event
>> > from ZMQ, but the payload of this event only contains "fd" (file
>> > descriptor I guess?). In such case is there a way to find the identity
>> of
>> > the peer (as set on the client side using the ZMQ_IDENTITY option)?
>> >
>> > Joel._______________________________________________
>> >
>> > ________________________________
>> >
>> > Declaración de confidencialidad: Este Mensaje esta destinado para el
>> uso de
>> > la o las personas o entidades a quien ha sido dirigido y puede contener
>> > información reservada y confidencial que no puede ser divulgada,
>> difundida,
>> > ni aprovechada en forma alguna. El uso no autorizado de la información
>> > contenida en este correo podrá ser sancionado de conformidad con la ley
>> > chilena. Si usted ha recibido este correo electrónico por error, le
>> pedimos
>> > eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
>> > remitente, respondiendo este mensaje. Disclosure: This Message is to be
>> used
>> > by the individual, individuals or entities that it is addressed to and
>> may
>> > include private and confidential information that may not be disclosed,
>> made
>> > public nor used in any way at all. Unauthorized use of the information
>> in
>> > this electronic mail message may be subject to the penalties set forth
>> by
>> > Chilean law. If you have received this electronic mail message in
>> error, we
>> > ask you to destroy the message and its attached file(s) and to
>> immediately
>> > notify the sender by answering this message.
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> ****
>>
>> ===========================================================****
>>
>> The information in this email is confidential, and is intended solely for the addressee(s). ****
>>
>> Access to this email by anyone else is unauthorized and therefore prohibited.  If you are ****
>>
>> not the intended recipient you are notified that disclosing, copying, distributing or taking ****
>>
>> any action in reliance on the contents of this information is strictly prohibited and may be unlawful.****
>>
>> ===========================================================****
>>
>>   ------------------------------
>>
>> Declaración de confidencialidad: Este Mensaje esta destinado para el uso
>> de la o las personas o entidades a quien ha sido dirigido y puede contener
>> información reservada y confidencial que no puede ser divulgada, difundida,
>> ni aprovechada en forma alguna. El uso no autorizado de la información
>> contenida en este correo podrá ser sancionado de conformidad con la ley
>> chilena. Si usted ha recibido este correo electrónico por error, le pedimos
>> eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
>> remitente, respondiendo este mensaje. Disclosure: This Message is to be
>> used by the individual, individuals or entities that it is addressed to and
>> may include private and confidential information that may not be disclosed,
>> made public nor used in any way at all. Unauthorized use of the information
>> in this electronic mail message may be subject to the penalties set forth
>> by Chilean law. If you have received this electronic mail message in error,
>> we ask you to destroy the message and its attached file(s) and to
>> immediately notify the sender by answering this message.
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/c3825d46/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/c3825d46/attachment.gif>


More information about the zeromq-dev mailing list