[zeromq-dev] Draft proposal for revamped ZMQ_MONITOR functionality

Pieter Hintjens ph at imatix.com
Tue Oct 1 18:43:51 CEST 2013


You should fork libzmq master, and when you have a minimal implementation,
make a pull request. We can refine it over many iterations then.


On Tue, Oct 1, 2013 at 6:12 PM, gonzalo diethelm <gdiethelm at dcv.cl> wrote:

>  Pieter, which branch should I use to take a whack at implementing this?**
> **
>
> ** **
>
> -- ****
>
> Gonzalo Diethelm****
>
> DCV Chile****
>
> ** **
>
> *From:* zeromq-dev-bounces at lists.zeromq.org [mailto:
> zeromq-dev-bounces at lists.zeromq.org] *On Behalf Of *gonzalo diethelm
> *Sent:* Tuesday, October 01, 2013 12:46 PM
>
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] Draft proposal for revamped ZMQ_MONITOR
> functionality****
>
>  ** **
>
> I think two frames would work for all event types:****
>
>  ****
>
> **1.      **Event type id: one of the ZMQ_EVENT_* constants, formatted as
> a hex or decimal number.****
>
> **2.      **Event payload, which depends on the event type, always
> formatted as multiple lines with “key=value” format.****
>
>  ****
>
> Right?****
>
>  ****
>
> -- ****
>
> Gonzalo Diethelm****
>
> DCV Chile****
>
>  ****
>
> *From:* zeromq-dev-bounces at lists.zeromq.org [
> mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>]
> *On Behalf Of *Lyle Thompson
> *Sent:* Tuesday, October 01, 2013 12:35 PM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] Draft proposal for revamped ZMQ_MONITOR
> functionality****
>
>  ****
>
> It would be good to use a framing mechanism so that event handlers don’t
> have to know all of the event types for them to read the right number of
> frames for each event. This would make it easier to ignore events, and
> allow adding new events without requiring message handlers to be upgraded.
> Maybe one of the bytes of the event id could be used? Or maybe there would
> always be one payload frame with delimited fields? Or throw in an empty
> sentinel frame, in the idiomatic fashion.****
>
>  ****
>
> Or did I miss something?****
>
>  ****
>
> Lyle****
>
>  ****
>
> *From:* zeromq-dev-bounces at lists.zeromq.org [
> mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>]
> *On Behalf Of *Pieter Hintjens
> *Sent:* Tuesday, October 01, 2013 7:33 AM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] Draft proposal for revamped ZMQ_MONITOR
> functionality****
>
>  ****
>
> Some thoughts on this...****
>
>  ****
>
> - I really like the idea of text events that can be easily parsed, and
> trivially printed for debugging****
>
> - Instead of socket numbers, we could use printable identities****
>
> - Instead of event numbers, we could use short string constants****
>
> - name=value pairs is a great idea****
>
>  ****
>
> -Pieter****
>
>  ****
>
> On Tue, Oct 1, 2013 at 4:09 PM, gonzalo diethelm <gdiethelm at dcv.cl> wrote:
> ****
>
> OK, this is what I am thinking, in a very rough first cut, of how
> ZMQ_MONITOR should work. Some notes follow. Comments are welcome.****
>
>  ****
>
> 1.       This proposal entails a change in the monitor API.****
>
> 2.       The purpose is to provide more event types and more room for
> growth, and ease support of this functionality from bindings, where dealing
> with binary data could be complex.****
>
> 3.       There is an assumption there won’t be much data transmitted with
> this mechanism, hence I have favored clarity and simplicity over efficiency.
> ****
>
> 4.       I tried to cover all possible event types likely to be of
> interest. There may be more. There may be better names for the event types.
> The proposed event types may already be covered in the current
> implementation.****
>
> 5.       Perhaps EVENT and SOCKET (see below) should be 64 bit quantities?
> ****
>
> 6.       Perhaps EVENT should be on a separate frame, as it is today?****
>
> 7.       I am not sure how to denote a (local or remote) endpoint. Should
> it be a char* (“tcp://192.168.11.5:5678”) or a 32 (64) bit value (file
> descriptor, pointer)?****
>
> 8.       To make it even easier, each line could have the format
> “key=value”. That would make it a breeze in some languages to read all
> lines into a dictionary data structure.****
>
>  ****
>
> Usage: as before, after calling zmq_socket_monitor() for certain sockets,
> on a different thread you would create a ZMQ_PAIR socket, connect that
> socket to the inproc endpoint specified with zmq_socket_monitor() and read
> messages in a loop.  Al messages are strings of the form:****
>
>  ****
>
>   EVENT****
>
>   SOCKET****
>
>   [additional lines]****
>
>  ****
>
> EVENT is an 8 byte, base 16, 32 bit numeric value identifying the event
> type that was triggered. Example: “45a2887f”.****
>
> SOCKET is an 8 byte, base 16, 32 bit numeric value identifying the local
> socket where the event was triggered. Example: “8721b4ce”.****
>
>  ****
>
> After these two fields there may be zero or more additional lines, with
> one field per line; each possible event type has its own predefined set of
> fields.****
>
>  ****
>
> Supported event types (and their additional fields) follow.  Those marked
> with (*) require a better, more concise name.****
>
>  ****
>
> ZMQ_EVENT_BIND_COMPLETED: current local socket bind on local endpoint
> completed****
>
>   local endpoint****
>
>  ****
>
> ZMQ_EVENT_UNBIND_COMPLETED: current local socket unbind from local
> endpoint completed****
>
>   local endpoint****
>
>  ****
>
> ZMQ_EVENT_CONNECT_COMPLETED: current local socket connection to remote
> endpoint completed****
>
>   remote endpoint****
>
>  ****
>
> ZMQ_EVENT_CONNECT_DELAYED: current local socket connection to remote
> endpoint delayed****
>
>   remote endpoint****
>
>  ****
>
> ZMQ_EVENT_CONNECT_RETRIED: current local socket connection to remote
> endpoint retried****
>
>   remote endpoint****
>
>  ****
>
> ZMQ_EVENT_ACCEPT_COMPLETED: remote endpoint connection to current local
> socket accepted****
>
>   remote endpoint****
>
>  ****
>
> ZMQ_EVENT_ACCEPT_FAILED: remote endpoint connection to current local
> socket failed****
>
>   remote endpoint****
>
>  ****
>
> (*) ZMQ_EVENT_LOCAL_DISCONNECT_COMPLETED: current local socket
> disconnected from remote endpoint****
>
>   remote endpoint****
>
>  ****
>
> (*) ZMQ_EVENT_REMOTE_DISCONNECT_COMPLETED: remote endpoint disconnected
> from current local socket****
>
>   remote endpoint****
>
>  ****
>
> ZMQ_EVENT_CLOSE_COMPLETED: current local socket close completed (useful
> for tracking progress in term())****
>
>   NONE****
>
>  ****
>
> ZMQ_EVENT_CLOSE_FAILED: current local socket close failed (useful for
> tracking progress in term())****
>
>   NONE****
>
>  ****
>
> ZMQ_EVENT_CORRUPTED: current local socket corruption detected. When does
> this happen?****
>
>   NONE****
>
>  ****
>
> ZMQ_EVENT_LINGERED: current local socket linger period expired****
>
>   linger period. Example: “1000” => 1 second****
>
>  ****
>
> (*) ZMQ_EVENT_REMOTE_IDENTIFIED: current local socket ascertained remote
> endpoint's identity****
>
>   remote endpoint****
>
>   remote identity (binary, 1 to 255 bytes, not starting with 0). Example:
> “68656c6c6f” => “hello”****
>
>  ****
>
> ZMQ_EVENT_SNDHWM_REACHED: send high water mark reached****
>
>   value of SNDHWM (number of messages). Example: “100” => 100 messages.***
> *
>
>  ****
>
> ZMQ_EVENT_RCVHWM_REACHED: receive high water mark reached****
>
>   value of RCVHWM (number of messages). Example: “200” => 200 messages.***
> *
>
>  ****
>
>  ****
>
> Maybe add events for these conditions as well:****
>
> subscription registration received from downstream****
>
> subscription unregistration received from downstream****
>
>  ****
>
> -- ****
>
> Gonzalo Diethelm****
>
> DCV Chile****
>
>  ****
>
> *From:* zeromq-dev-bounces at lists.zeromq.org [mailto:
> zeromq-dev-bounces at lists.zeromq.org] *On Behalf Of *
> Richard_Newton at waters.com
> *Sent:* Saturday, September 28, 2013 7:03 AM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor****
>
>  ****
>
> Formalizing the protocol would be good, but a default endpoint would make
> it difficult to monitor 2 different sockets from 2 different places in code
> would it not?  Not sure if anyone uses this use case but I could imagine a
> large zeromq based application might want to.
>
> Ric.
>
>
> [image: Inactive hide details for "Pieter Hintjens" ---28/09/2013 09:50:33
> AM---These sound like good ideas. I think we lack documentat]"Pieter
> Hintjens" ---28/09/2013 09:50:33 AM---These sound like good ideas. I
> think we lack documentation on the event interface, and it could be w
>
> From: "Pieter Hintjens" <ph at imatix.com>
> To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org>,
> Date: 28/09/2013 09:50 AM
> Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
> Sent by: zeromq-dev-bounces at lists.zeromq.org****
>   ------------------------------
>
>
>
>
> These sound like good ideas. I think we lack documentation on the
> event interface, and it could be worth wrapping in CZMQ too.
>
> I think formalizing the internal protocol would be good, and using a
> default inproc endpoint as we do with ZAP could make it simpler.
>
> On Fri, Sep 27, 2013 at 8:57 PM, gonzalo diethelm <gdiethelm at dcv.cl>
> wrote:
> > I have been looking into the monitor mechanism and I think there are
> several
> > shortcomings, that I would like to address. I would like some opinions
> on
> > these points before diving into the code.
> >
> >
> >
> > 1.       We really need two separate “disconnected” events: one
> > “local_close” for when the local socket has disconnected from its remote
> > peers, another “remote_disconnected” for when one (out of possibly many)
> of
> > the socket’s remote peers has disconnected from us. In the second case,
> it
> > is necessary to identify the remote peer (by its address?).
> >
> > 2.       We should also add a new type of event (as Doron Somech
> suggested
> > half a year ago) that will trigger whenever the peer’s identity is
> > ascertained by the local socket. This would allow us to then identify
> which
> > was the peer that produced a “remote_disconnected” event.
> >
> > 3.       I think (but have not thought this through) it would be easier
> for
> > bindings if all the event data (except perhaps the event id) was
> formatted
> > as ASCII strings, otherwise it is necessary to interpret binary data in
> > other languages.
> >
> > 4.       I think we should switch from 16 to 32 bits for the event mask,
> we
> > have already used up the first 10 bits.
> >
> >
> >
> > Best regards,
> >
> >
> >
> > --
> >
> > Gonzalo Diethelm
> >
> > DCV Chile
> >
> >
> >
> > From: zeromq-dev-bounces at lists.zeromq.org
> > [mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>]
> On Behalf Of Joel Lauener
> > Sent: Wednesday, August 07, 2013 9:53 AM
> > To: zeromq-dev at lists.zeromq.org
> >
> >
> > Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
> >
> >
> >
> > Hi Richard,
> >
> >
> >
> > Thanks for your answer. I already receive the address but in the
> particular
> > case of a router socket it's not that useful. I receive something like
> > 'tcp://0.0.0.0:2000' which is the address to which the socket is bound.
> >
> >
> >
> > My question was if there is a way to know what is the identity of the
> peer
> > that has been disconnected (as set on the client side with the
> ZMQ_IDENTITY
> > option). There are many connections behind a router. When I receive a
> > disconnect event I need to know to which connection it belongs in order
> to
> > take the correct action on the server (callback user code, cleanup
> resource,
> > ..).
> >
> >
> >
> > Cheers,
> >
> > Joel.
> >
> >
> >
> > Date: Mon, 5 Aug 2013 11:12:34 +0100
> > From: Richard_Newton at waters.com
> > Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
> > To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org>
> > Cc: zeromq-dev-bounces at lists.zeromq.org
> > Message-ID:
> >
> > <OF242DDFB2.16EF4ADC-ON80257BBE.0037B5C6-80257BBE.00381560 at waters.com>
> > Content-Type: text/plain; charset="us-ascii"
> >
> >
> > Hi Joel,
> >
> > The monitor event mechanism uses 2 sends to communicate, the first
> message
> > contains the event details and fd, the second will contain the address,
> so
> > you need to do a second receive to get the address.
> >
> > Apparently this was to make it easier for language bindings.
> >
> > Ric.
> >
> >
> >
> >
> > From:   "Joel Lauener" <Joel.Lauener at cern.ch>
> > To:     "zeromq-dev at lists.zeromq.org" <zeromq-dev at lists.zeromq.org>
> > Date:   05/08/2013 11:02 AM
> > Subject:        [zeromq-dev] ROUTER socket and zmq_socket_monitor
> > Sent by:        zeromq-dev-bounces at lists.zeromq.org
> >
> >
> >
> > Hi,
> >
> > Currently I'm trying to setup a monitor on a ROUTER socket to detect
> clients
> > disconnections. This system is put in place in addition to an heartbeat
> > mechanism in order to provide faster disconnection detection on the
> server
> > side.
> >
> > My problem now is that I effectively receive a ZMQ_EVENT_DISCONNECTED
> event
> > from ZMQ, but the payload of this event only contains "fd" (file
> > descriptor I guess?). In such case is there a way to find the identity of
> > the peer (as set on the client side using the ZMQ_IDENTITY option)?
> >
> > Joel._______________________________________________
> >
> > ________________________________
> >
> > Declaración de confidencialidad: Este Mensaje esta destinado para el uso
> de
> > la o las personas o entidades a quien ha sido dirigido y puede contener
> > información reservada y confidencial que no puede ser divulgada,
> difundida,
> > ni aprovechada en forma alguna. El uso no autorizado de la información
> > contenida en este correo podrá ser sancionado de conformidad con la ley
> > chilena. Si usted ha recibido este correo electrónico por error, le
> pedimos
> > eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
> > remitente, respondiendo este mensaje. Disclosure: This Message is to be
> used
> > by the individual, individuals or entities that it is addressed to and
> may
> > include private and confidential information that may not be disclosed,
> made
> > public nor used in any way at all. Unauthorized use of the information in
> > this electronic mail message may be subject to the penalties set forth by
> > Chilean law. If you have received this electronic mail message in error,
> we
> > ask you to destroy the message and its attached file(s) and to
> immediately
> > notify the sender by answering this message.
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev****
>
> ===========================================================****
>
> The information in this email is confidential, and is intended solely for the addressee(s). ****
>
> Access to this email by anyone else is unauthorized and therefore prohibited.  If you are ****
>
> not the intended recipient you are notified that disclosing, copying, distributing or taking ****
>
> any action in reliance on the contents of this information is strictly prohibited and may be unlawful.****
>
> ===========================================================****
>
>    ------------------------------
>
> Declaración de confidencialidad: Este Mensaje esta destinado para el uso
> de la o las personas o entidades a quien ha sido dirigido y puede contener
> información reservada y confidencial que no puede ser divulgada, difundida,
> ni aprovechada en forma alguna. El uso no autorizado de la información
> contenida en este correo podrá ser sancionado de conformidad con la ley
> chilena. Si usted ha recibido este correo electrónico por error, le pedimos
> eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
> remitente, respondiendo este mensaje. Disclosure: This Message is to be
> used by the individual, individuals or entities that it is addressed to and
> may include private and confidential information that may not be disclosed,
> made public nor used in any way at all. Unauthorized use of the information
> in this electronic mail message may be subject to the penalties set forth
> by Chilean law. If you have received this electronic mail message in error,
> we ask you to destroy the message and its attached file(s) and to
> immediately notify the sender by answering this message.****
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev****
>
>  ****
>   ------------------------------
>
> Declaración de confidencialidad: Este Mensaje esta destinado para el uso
> de la o las personas o entidades a quien ha sido dirigido y puede contener
> información reservada y confidencial que no puede ser divulgada, difundida,
> ni aprovechada en forma alguna. El uso no autorizado de la información
> contenida en este correo podrá ser sancionado de conformidad con la ley
> chilena. Si usted ha recibido este correo electrónico por error, le pedimos
> eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
> remitente, respondiendo este mensaje.****
>
> Disclosure: This Message is to be used by the individual, individuals or
> entities that it is addressed to and may include private and confidential
> information that may not be disclosed, made public nor used in any way at
> all. Unauthorized use of the information in this electronic mail message
> may be subject to the penalties set forth by Chilean law. If you have
> received this electronic mail message in error, we ask you to destroy the
> message and its attached file(s) and to immediately notify the sender by
> answering this message.****
>
> ------------------------------
>
> Declaración de confidencialidad: Este Mensaje esta destinado para el uso
> de la o las personas o entidades a quien ha sido dirigido y puede contener
> información reservada y confidencial que no puede ser divulgada, difundida,
> ni aprovechada en forma alguna. El uso no autorizado de la información
> contenida en este correo podrá ser sancionado de conformidad con la ley
> chilena. Si usted ha recibido este correo electrónico por error, le pedimos
> eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
> remitente, respondiendo este mensaje. Disclosure: This Message is to be
> used by the individual, individuals or entities that it is addressed to and
> may include private and confidential information that may not be disclosed,
> made public nor used in any way at all. Unauthorized use of the information
> in this electronic mail message may be subject to the penalties set forth
> by Chilean law. If you have received this electronic mail message in error,
> we ask you to destroy the message and its attached file(s) and to
> immediately notify the sender by answering this message.
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/9cd145e4/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/9cd145e4/attachment.gif>


More information about the zeromq-dev mailing list