[zeromq-dev] Draft proposal for revamped ZMQ_MONITOR functionality

Lyle Thompson lthompson at ixiacom.com
Tue Oct 1 17:34:39 CEST 2013 

It would be good to use a framing mechanism so that event handlers don't have to know all of the event types for them to read the right number of frames for each event. This would make it easier to ignore events, and allow adding new events without requiring message handlers to be upgraded. Maybe one of the bytes of the event id could be used? Or maybe there would always be one payload frame with delimited fields? Or throw in an empty sentinel frame, in the idiomatic fashion.

Or did I miss something?

Lyle

From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: Tuesday, October 01, 2013 7:33 AM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Draft proposal for revamped ZMQ_MONITOR functionality

Some thoughts on this...

- I really like the idea of text events that can be easily parsed, and trivially printed for debugging
- Instead of socket numbers, we could use printable identities
- Instead of event numbers, we could use short string constants
- name=value pairs is a great idea

-Pieter

On Tue, Oct 1, 2013 at 4:09 PM, gonzalo diethelm <gdiethelm at dcv.cl<mailto:gdiethelm at dcv.cl>> wrote:
OK, this is what I am thinking, in a very rough first cut, of how ZMQ_MONITOR should work. Some notes follow. Comments are welcome.


1.       This proposal entails a change in the monitor API.

2.       The purpose is to provide more event types and more room for growth, and ease support of this functionality from bindings, where dealing with binary data could be complex.

3.       There is an assumption there won't be much data transmitted with this mechanism, hence I have favored clarity and simplicity over efficiency.

4.       I tried to cover all possible event types likely to be of interest. There may be more. There may be better names for the event types. The proposed event types may already be covered in the current implementation.

5.       Perhaps EVENT and SOCKET (see below) should be 64 bit quantities?

6.       Perhaps EVENT should be on a separate frame, as it is today?

7.       I am not sure how to denote a (local or remote) endpoint. Should it be a char* ("tcp://192.168.11.5:5678") or a 32 (64) bit value (file descriptor, pointer)?

8.       To make it even easier, each line could have the format "key=value". That would make it a breeze in some languages to read all lines into a dictionary data structure.

Usage: as before, after calling zmq_socket_monitor() for certain sockets, on a different thread you would create a ZMQ_PAIR socket, connect that socket to the inproc endpoint specified with zmq_socket_monitor() and read messages in a loop.  Al messages are strings of the form:

  EVENT
  SOCKET
  [additional lines]

EVENT is an 8 byte, base 16, 32 bit numeric value identifying the event type that was triggered. Example: "45a2887f".
SOCKET is an 8 byte, base 16, 32 bit numeric value identifying the local socket where the event was triggered. Example: "8721b4ce".

After these two fields there may be zero or more additional lines, with one field per line; each possible event type has its own predefined set of fields.

Supported event types (and their additional fields) follow.  Those marked with (*) require a better, more concise name.

ZMQ_EVENT_BIND_COMPLETED: current local socket bind on local endpoint completed
  local endpoint

ZMQ_EVENT_UNBIND_COMPLETED: current local socket unbind from local endpoint completed
  local endpoint

ZMQ_EVENT_CONNECT_COMPLETED: current local socket connection to remote endpoint completed
  remote endpoint

ZMQ_EVENT_CONNECT_DELAYED: current local socket connection to remote endpoint delayed
  remote endpoint

ZMQ_EVENT_CONNECT_RETRIED: current local socket connection to remote endpoint retried
  remote endpoint

ZMQ_EVENT_ACCEPT_COMPLETED: remote endpoint connection to current local socket accepted
  remote endpoint

ZMQ_EVENT_ACCEPT_FAILED: remote endpoint connection to current local socket failed
  remote endpoint

(*) ZMQ_EVENT_LOCAL_DISCONNECT_COMPLETED: current local socket disconnected from remote endpoint
  remote endpoint

(*) ZMQ_EVENT_REMOTE_DISCONNECT_COMPLETED: remote endpoint disconnected from current local socket
  remote endpoint

ZMQ_EVENT_CLOSE_COMPLETED: current local socket close completed (useful for tracking progress in term())
  NONE

ZMQ_EVENT_CLOSE_FAILED: current local socket close failed (useful for tracking progress in term())
  NONE

ZMQ_EVENT_CORRUPTED: current local socket corruption detected. When does this happen?
  NONE

ZMQ_EVENT_LINGERED: current local socket linger period expired
  linger period. Example: "1000" => 1 second

(*) ZMQ_EVENT_REMOTE_IDENTIFIED: current local socket ascertained remote endpoint's identity
  remote endpoint
  remote identity (binary, 1 to 255 bytes, not starting with 0). Example: "68656c6c6f" => "hello"

ZMQ_EVENT_SNDHWM_REACHED: send high water mark reached
  value of SNDHWM (number of messages). Example: "100" => 100 messages.

ZMQ_EVENT_RCVHWM_REACHED: receive high water mark reached
  value of RCVHWM (number of messages). Example: "200" => 200 messages.


Maybe add events for these conditions as well:
subscription registration received from downstream
subscription unregistration received from downstream

--
Gonzalo Diethelm
DCV Chile

From: zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org> [mailto:zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org>] On Behalf Of Richard_Newton at waters.com<mailto:Richard_Newton at waters.com>
Sent: Saturday, September 28, 2013 7:03 AM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor


Formalizing the protocol would be good, but a default endpoint would make it difficult to monitor 2 different sockets from 2 different places in code would it not?  Not sure if anyone uses this use case but I could imagine a large zeromq based application might want to.

Ric.


[Inactive hide details for "Pieter Hintjens" ---28/09/2013 09:50:33 AM---These sound like good ideas. I think we lack documentat]"Pieter Hintjens" ---28/09/2013 09:50:33 AM---These sound like good ideas. I think we lack documentation on the event interface, and it could be w

From: "Pieter Hintjens" <ph at imatix.com<mailto:ph at imatix.com>>
To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>>,
Date: 28/09/2013 09:50 AM
Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
Sent by: zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org>

________________________________



These sound like good ideas. I think we lack documentation on the
event interface, and it could be worth wrapping in CZMQ too.

I think formalizing the internal protocol would be good, and using a
default inproc endpoint as we do with ZAP could make it simpler.

On Fri, Sep 27, 2013 at 8:57 PM, gonzalo diethelm <gdiethelm at dcv.cl<mailto:gdiethelm at dcv.cl>> wrote:
> I have been looking into the monitor mechanism and I think there are several
> shortcomings, that I would like to address. I would like some opinions on
> these points before diving into the code.
>
>
>
> 1.       We really need two separate "disconnected" events: one
> "local_close" for when the local socket has disconnected from its remote
> peers, another "remote_disconnected" for when one (out of possibly many) of
> the socket's remote peers has disconnected from us. In the second case, it
> is necessary to identify the remote peer (by its address?).
>
> 2.       We should also add a new type of event (as Doron Somech suggested
> half a year ago) that will trigger whenever the peer's identity is
> ascertained by the local socket. This would allow us to then identify which
> was the peer that produced a "remote_disconnected" event.
>
> 3.       I think (but have not thought this through) it would be easier for
> bindings if all the event data (except perhaps the event id) was formatted
> as ASCII strings, otherwise it is necessary to interpret binary data in
> other languages.
>
> 4.       I think we should switch from 16 to 32 bits for the event mask, we
> have already used up the first 10 bits.
>
>
>
> Best regards,
>
>
>
> --
>
> Gonzalo Diethelm
>
> DCV Chile
>
>
>
> From: zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org>
> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Joel Lauener
> Sent: Wednesday, August 07, 2013 9:53 AM
> To: zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
>
>
> Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
>
>
>
> Hi Richard,
>
>
>
> Thanks for your answer. I already receive the address but in the particular
> case of a router socket it's not that useful. I receive something like
> 'tcp://0.0.0.0:2000<http://0.0.0.0:2000>' which is the address to which the socket is bound.
>
>
>
> My question was if there is a way to know what is the identity of the peer
> that has been disconnected (as set on the client side with the ZMQ_IDENTITY
> option). There are many connections behind a router. When I receive a
> disconnect event I need to know to which connection it belongs in order to
> take the correct action on the server (callback user code, cleanup resource,
> ..).
>
>
>
> Cheers,
>
> Joel.
>
>
>
> Date: Mon, 5 Aug 2013 11:12:34 +0100
> From: Richard_Newton at waters.com<mailto:Richard_Newton at waters.com>
> Subject: Re: [zeromq-dev] ROUTER socket and zmq_socket_monitor
> To: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>>
> Cc: zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org>
> Message-ID:
>
> <OF242DDFB2.16EF4ADC-ON80257BBE.0037B5C6-80257BBE.00381560 at waters.com<mailto:OF242DDFB2.16EF4ADC-ON80257BBE.0037B5C6-80257BBE.00381560 at waters.com>>
> Content-Type: text/plain; charset="us-ascii"
>
>
> Hi Joel,
>
> The monitor event mechanism uses 2 sends to communicate, the first message
> contains the event details and fd, the second will contain the address, so
> you need to do a second receive to get the address.
>
> Apparently this was to make it easier for language bindings.
>
> Ric.
>
>
>
>
> From:   "Joel Lauener" <Joel.Lauener at cern.ch<mailto:Joel.Lauener at cern.ch>>
> To:     "zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>" <zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>>
> Date:   05/08/2013 11:02 AM
> Subject:        [zeromq-dev] ROUTER socket and zmq_socket_monitor
> Sent by:        zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org>
>
>
>
> Hi,
>
> Currently I'm trying to setup a monitor on a ROUTER socket to detect clients
> disconnections. This system is put in place in addition to an heartbeat
> mechanism in order to provide faster disconnection detection on the server
> side.
>
> My problem now is that I effectively receive a ZMQ_EVENT_DISCONNECTED event
> from ZMQ, but the payload of this event only contains "fd" (file
> descriptor I guess?). In such case is there a way to find the identity of
> the peer (as set on the client side using the ZMQ_IDENTITY option)?
>
> Joel._______________________________________________
>
> ________________________________
>
> Declaración de confidencialidad: Este Mensaje esta destinado para el uso de
> la o las personas o entidades a quien ha sido dirigido y puede contener
> información reservada y confidencial que no puede ser divulgada, difundida,
> ni aprovechada en forma alguna. El uso no autorizado de la información
> contenida en este correo podrá ser sancionado de conformidad con la ley
> chilena. Si usted ha recibido este correo electrónico por error, le pedimos
> eliminarlo junto con los archivos adjuntos y avisar inmediatamente al
> remitente, respondiendo este mensaje. Disclosure: This Message is to be used
> by the individual, individuals or entities that it is addressed to and may
> include private and confidential information that may not be disclosed, made
> public nor used in any way at all. Unauthorized use of the information in
> this electronic mail message may be subject to the penalties set forth by
> Chilean law. If you have received this electronic mail message in error, we
> ask you to destroy the message and its attached file(s) and to immediately
> notify the sender by answering this message.
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

===========================================================

The information in this email is confidential, and is intended solely for the addressee(s).

Access to this email by anyone else is unauthorized and therefore prohibited.  If you are

not the intended recipient you are notified that disclosing, copying, distributing or taking

any action in reliance on the contents of this information is strictly prohibited and may be unlawful.

===========================================================

________________________________
Declaración de confidencialidad: Este Mensaje esta destinado para el uso de la o las personas o entidades a quien ha sido dirigido y puede contener información reservada y confidencial que no puede ser divulgada, difundida, ni aprovechada en forma alguna. El uso no autorizado de la información contenida en este correo podrá ser sancionado de conformidad con la ley chilena. Si usted ha recibido este correo electrónico por error, le pedimos eliminarlo junto con los archivos adjuntos y avisar inmediatamente al remitente, respondiendo este mensaje. Disclosure: This Message is to be used by the individual, individuals or entities that it is addressed to and may include private and confidential information that may not be disclosed, made public nor used in any way at all. Unauthorized use of the information in this electronic mail message may be subject to the penalties set forth by Chilean law. If you have received this electronic mail message in error, we ask you to destroy the message and its attached file(s) and to immediately notify the sender by answering this message.

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/8b8fe42b/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 105 bytes
Desc: image001.gif
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131001/8b8fe42b/attachment.gif>

More information about the zeromq-dev mailing list