[zeromq-dev] Draft proposal for revamped ZMQ_MONITOR functionality

Pieter Hintjens ph at imatix.com
Tue Oct 1 16:32:49 CEST 2013


Some thoughts on this...

- I really like the idea of text events that can be easily parsed, and
trivially printed for debugging
- Instead of socket numbers, we could use printable identities
- Instead of event numbers, we could use short string constants
- name=value pairs is a great idea

-Pieter


On Tue, Oct 1, 2013 at 4:09 PM, gonzalo diethelm <gdiethelm at dcv.cl> wrote:

>  OK, this is what I am thinking, in a very rough first cut, of how
> ZMQ_MONITOR should work. Some notes follow. Comments are welcome.****
>
> ** **
>
> **1.       **This proposal entails a change in the monitor API.****
>
> **2.       **The purpose is to provide more event types and more room for
> growth, and ease support of this functionality from bindings, where dealing
> with binary data could be complex.****
>
> **3.       **There is an assumption there won’t be much data transmitted
> with this mechanism, hence I have favored clarity and simplicity over
> efficiency.****
>
> **4.       **I tried to cover all possible event types likely to be of
> interest. There may be more. There may be better names for the event types.
> The proposed event types may already be covered in the current
> implementation.****
>
> **5.       **Perhaps EVENT and SOCKET (see below) should be 64 bit
> quantities?****
>
> **6.       **Perhaps EVENT should be on a separate frame, as it is today?*
> ***
>
> **7.       **I am not sure how to denote a (local or remote) endpoint.
> Should it be a char* (“tcp://192.168.11.5:5678”) or a 32 (64) bit value
> (file descriptor, pointer)?****
>
> **8.       **To make it even easier, each line could have the format
> “key=value”. That would make it a breeze in some languages to read all
> lines into a dictionary data structure.****
>
> ** **
>
> Usage: as before, after calling zmq_socket_monitor() for certain sockets,
> on a different thread you would create a ZMQ_PAIR socket, connect that
> socket to the inproc endpoint specified with zmq_socket_monitor() and read
> messages in a loop.  Al messages are strings of the form:****
>
> ** **
>
>   EVENT****
>
>   SOCKET****
>
>   [additional lines]****
>
> ** **
>
> EVENT is an 8 byte, base 16, 32 bit numeric value identifying the event
> type that was triggered. Example: “45a2887f”.****
>
> SOCKET is an 8 byte, base 16, 32 bit numeric value identifying the local
> socket where the event was triggered. Example: “8721b4ce”.****
>
> ** **
>
> After these two fields there may be zero or more additional lines, with
> one field per line; each possible event type has its own predefined set of
> fields.****
>
> ** **
>
> Supported event types (and their additional fields) follow.  Those marked
> with (*) require a better, more concise name.****
>
> ** **
>
> ZMQ_EVENT_BIND_COMPLETED: current local socket bind on local endpoint
> completed****
>
>   local endpoint****
>
> ** **
>
> ZMQ_EVENT_UNBIND_COMPLETED: current local socket unbind from local
> endpoint completed****
>
>   local endpoint****
>
> ** **
>
> ZMQ_EVENT_CONNECT_COMPLETED: current local socket connection to remote
> endpoint completed****
>
>   remote endpoint****
>
> ** **
>
> ZMQ_EVENT_CONNECT_DELAYED: current local socket connection to remote
> endpoint delayed****
>
>   remote endpoint****
>
> ** **
>
> ZMQ_EVENT_CONNECT_RETRIED: current local socket connection to remote
> endpoint retried****
>
>   remote endpoint****
>
> ** **
>
> ZMQ_EVENT_ACCEPT_COMPLETED: remote endpoint connection to current local
> socket accepted****
>
>   remote endpoint****
>
> ** **
>
> ZMQ_EVENT_ACCEPT_FAILED: remote endpoint connection to current local
> socket failed****
>
>   remote endpoint****
>
> ** **
>
> (*) ZMQ_EVENT_LOCAL_DISCONNECT_COMPLETED: current local socket
> disconnected from remote endpoint****
>
>   remote endpoint****
>
> ** **
>
> (*) ZMQ_EVENT_REMOTE_DISCONNECT_COMPLETED: remote endpoint disconnected
> from current local socket****
>
>   remote endpoint****
>
> ** **
>
> ZMQ_EVENT_CLOSE_COMPLETED: current local socket close completed (useful
> for tracking progress in term())****
>
>   NONE****
>
> ** **
>
> ZMQ_EVENT_CLOSE_FAILED: current local socket close failed (useful for
> tracking progress in term())****
>
>   NONE****
>
> ** **
>
> ZMQ_EVENT_CORRUPTED: current local socket corruption detected. When does
> this happen?****
>
>   NONE****
>
> ** **
>
> ZMQ_EVENT_LINGERED: current local socket linger period expired****
>
>   linger period. Example: “1000” => 1 second****
>
> ** **
>
> (*) ZMQ_EVENT_REMOTE_IDENTIFIED: current local socket ascertained remote
> endpoint's identity****
>
>   remote endpoint****
>
>   remote identity (binary, 1 to 255 bytes, not starting with 0). Example:
> “68656c6c6f” => “hello”****
>
> ** **
>
> ZMQ_EVENT_SNDHWM_REACHED: send high water mark reached****
>
>   value of SNDHWM (number of messages). Example: “100” => 100 messages.***
> *
>
> ** **
>
> ZMQ_EVENT_RCVHWM_REACHED: receive high water mark reached****
>
>   value of RCVHWM (number of messages). Example: “200” => 200 messages.***
> *
>
> ** **
>
> ** **
>
> Maybe add events for these conditions as well:****
>
> subscription registration received from downstream****
>
> subscription unregistration received from downstream****
>
> ** **
>
> -- ****
>
> Gonzalo Diethelm****
>
> DCV Chile****
>
> ** **
>
> *From:* zeromq-dev-bounces at lists.zeromq.org [mailto:
> zeromq-dev-bounces at lists.zeromq.org] *On Behalf Of *
> Richard_Newton at waters.com
> *Sent:* Saturday, September 28, 2013 7:03 AM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor****
>
> ** **
>
> Formalizing the protocol would be good, but a default endpoint would make
> it difficult to monitor 2 different sockets from 2 different places in code
> would it not?  Not sure if anyone uses this use case but I could imagine a
> large zeromq based application might want to.
>
> Ric.
>
>
> [image: Inactive hide details for "Pieter Hintjens" ---28/09/2013 09:50:33
> AM---These sound like good ideas. I think we lack documentat]"Pieter
> Hintjens" ---28/09/2013 09:50:33 AM---These sound like good ideas. I
> think we lack documentation on the event interface, and it could be w
>
> From: "Pieter Hintjens" <ph at imatix.com>
> To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org>,
> Date: 28/09/2013 09:50 AM
> Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
> Sent by: zeromq-dev-bounces at lists.zeromq.org****
>  ------------------------------
>
>
>
>
> These sound like good ideas. I think we lack documentation on the
> event interface, and it could be worth wrapping in CZMQ too.
>
> I think formalizing the internal protocol would be good, and using a
> default inproc endpoint as we do with ZAP could make it simpler.
>
> On Fri, Sep 27, 2013 at 8:57 PM, gonzalo diethelm <gdiethelm at dcv.cl>
> wrote:
> > I have been looking into the monitor mechanism and I think there are
> several
> > shortcomings, that I would like to address. I would like some opinions
> on
> > these points before diving into the code.
> >
> >
> >
> > 1.       We really need two separate “disconnected” events: one
> > “local_close” for when the local socket has disconnected from its remote
> > peers, another “remote_disconnected” for when one (out of possibly many)
> of
> > the socket’s remote peers has disconnected from us. In the second case,
> it
> > is necessary to identify the remote peer (by its address?).
> >
> > 2.       We should also add a new type of event (as Doron Somech
> suggested
> > half a year ago) that will trigger whenever the peer’s identity is
> > ascertained by the local socket. This would allow us to then identify
> which
> > was the peer that produced a “remote_disconnected” event.
> >
> > 3.       I think (but have not thought this through) it would be easier
> for
> > bindings if all the event data (except perhaps the event id) was
> formatted
> > as ASCII strings, otherwise it is necessary to interpret binary data in
> > other languages.
> >
> > 4.       I think we should switch from 16 to 32 bits for the event mask,
> we
> > have already used up the first 10 bits.
> >
> >
> >
> > Best regards,
> >
> >
> >
> > --
> >
> > Gonzalo Diethelm
> >
> > DCV Chile
> >
> >
> >
> > From: zeromq-dev-bounces at lists.zeromq.org
> > [mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>]
> On Behalf Of Joel Lauener
> > Sent: Wednesday, August 07, 2013 9:53 AM
> > To: zeromq-dev at lists.zeromq.org
> >
> >
> > Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
> >
> >
> >
> > Hi Richard,
> >
> >
> >
> > Thanks for your answer. I already receive the address but in the
> particular
> > case of a router socket it's not that useful. I receive something like
> > 'tcp://0.0.0.0:2000' which is the address to which the socket is bound.
> >
> >
> >
> > My question was if there is a way to know what is the identity of the
> peer
> > that has been disconnected (as set on the client side with the
> ZMQ_IDENTITY
> > option). There are many connections behind a router. When I receive a
> > disconnect event I need to know to which connection it belongs in order
> to
> > take the correct action on the server (callback user code, cleanup
> resource,
> > ..).
> >
> >
> >
> > Cheers,
> >
> > Joel.
> >
> >
> >
> > Date: Mon, 5 Aug 2013 11:12:34 +0100
> > From: Richard_Newton at waters.com
> > Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
> > To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org>
> > Cc: zeromq-dev-bounces at lists.zeromq.org
> > Message-ID:
> >
> > <OF242DDFB2.16EF4ADC-ON80257BBE.0037B5C6-80257BBE.00381560 at waters.com>
> > Content-Type: text/plain; charset="us-ascii"
> >
> >
> > Hi Joel,
> >
> > The monitor event mechanism uses 2 sends to communicate, the first
> message
> > contains the event details and fd, the second will contain the address,
> so
> > you need to do a second receive to get the address.
> >
> > Apparently this was to make it easier for language bindings.
> >
> > Ric.
> >
> >
> >
> >
> > From:   "Joel Lauener" <Joel.Lauener at cern.ch>
> > To:     "zeromq-dev at lists.zeromq.org" <zeromq-dev at lists.zeromq.org>
> > Date:   05/08/2013 11:02 AM
> > Subject:        [zeromq-dev] ROUTER socket and zmq_socket_monitor
> > Sent by:        zeromq-dev-bounces at lists.zeromq.org
> >
> >
> >
> > Hi,
> >
> > Currently I'm trying to setup a monitor on a ROUTER socket to detect
> clients
> > disconnections. This system is put in place in addition to an heartbeat
> > mechanism in order to provide faster disconnection detection on the
> server
> > side.
> >
> > My problem now is that I effectively receive a ZMQ_EVENT_DISCONNECTED
> event
> > from ZMQ, but the payload of this event only contains "fd" (file
> > descriptor I guess?). In such case is there a way to find the identity of
> > the peer (as set on the client side using the ZMQ_IDENTITY option)?
> >
> > Joel._______________________________________________
> >
> > ________________________________
> >
> > Declaración de confidencialidad: Este Mensaje esta destinado para el uso
> de
> > la o las personas o entidades a quien ha sido dirigido y puede contener
> > información reservada y confidencial que no puede ser divulgada,
> difundida,
> > ni aprovechada en forma alguna. El uso no autorizado de la información
> > contenida en este correo podrá ser sancionado de conformidad con la ley
> > chilena. Si usted ha recibido este correo electrónico por error, le
> pedimos
> > eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
> > remitente, respondiendo este mensaje. Disclosure: This Message is to be
> used
> > by the individual, individuals or entities that it is addressed to and
> may
> > include private and confidential information that may not be disclosed,
> made
> > public nor used in any way at all. Unauthorized use of the information in
> > this electronic mail message may be subject to the penalties set forth by
> > Chilean law. If you have received this electronic mail message in error,
> we
> > ask you to destroy the message and its attached file(s) and to
> immediately
> > notify the sender by answering this message.
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> ****
>
> ===========================================================****
>
> The information in this email is confidential, and is intended solely for the addressee(s). ****
>
> Access to this email by anyone else is unauthorized and therefore prohibited.  If you are ****
>
> not the intended recipient you are notified that disclosing, copying, distributing or taking ****
>
> any action in reliance on the contents of this information is strictly prohibited and may be unlawful.****
>
> ===========================================================****
>
>   ------------------------------
>
> Declaración de confidencialidad: Este Mensaje esta destinado para el uso
> de la o las personas o entidades a quien ha sido dirigido y puede contener
> información reservada y confidencial que no puede ser divulgada, difundida,
> ni aprovechada en forma alguna. El uso no autorizado de la información
> contenida en este correo podrá ser sancionado de conformidad con la ley
> chilena. Si usted ha recibido este correo electrónico por error, le pedimos
> eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
> remitente, respondiendo este mensaje. Disclosure: This Message is to be
> used by the individual, individuals or entities that it is addressed to and
> may include private and confidential information that may not be disclosed,
> made public nor used in any way at all. Unauthorized use of the information
> in this electronic mail message may be subject to the penalties set forth
> by Chilean law. If you have received this electronic mail message in error,
> we ask you to destroy the message and its attached file(s) and to
> immediately notify the sender by answering this message.
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/29d37d98/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/29d37d98/attachment.gif>


More information about the zeromq-dev mailing list