[zeromq-dev] RFC: PPPP - Paranoid Pirate Publishing Protocol

[Goswin von Brederlow]

On Thu, Jul 03, 2014 at 11:50:44AM +0200, Pieter Hintjens wrote:
> On Thu, Jul 3, 2014 at 10:35 AM, Goswin von Brederlow <goswin-v-b at web.de> wrote:
> 
> > I can think of 3 ways to implement this (across all socket types):
> 
> Do we want this across all socket types?
> 
> For ROUTER, certainly. Right now we're hacking this using ZMQ_ROUTER_PROBE.
> 
> For other socket types, it seems to break the abstraction.
> 
> > 1) Sending a message with identity (or command?) flag set containing
> > the identity and wether it is a connect or disconnect as frames. For
> > backwards compatibility that should only be send when a socket option
> > is set, default to off.
> 
> Yes.
> 
> > 2) Include the identity in the DISCONNECT event on the monitor socket.
> > 3) Add 2 new monitoring events: IDENTITY_SET and IDENTITY_DISCONNECTED.
> 
> This was the original argument against adding monitoring at all, that
> it would be abused for topology purposes. Once you start to give the
> application knowledge of the topology, you break the ZeroMQ
> abstraction and you kill scalability. (This was Sustrik's argument,
> and I mostly accept it.)
> 
> More pragmatically, you *cannot* safely use monitoring for such
> things. Monitoring events are asynch and processed out of band. You
> cannot sanely synchronize monitoring events with message flow, except
> by injecting them into the message flow itself, and then you get
> option (1).
> 
> The simplest backwards compatible solution would be:
> 
> 1. use a socket option to enable connect/disconnect events
> 2. deliver these events as messages, on the socket
> 3. use a message format that is easy to filter, e.g. size=2
> 4. use a message flag, ZMQ_EVENT, via zmq_msg_get
> 5. enumerate the events, 1=CONNECT, 2=DISCONNECT, etc.
> 6. optionally, allow more parts to follow, depending on the event type.
> 
> This could be implemented for all socket types. There are other events
> we could conceivably add, and we could fix XPUB to use events instead
> of its magic subscribe/unsubscribe messages.

ZMQ_STREAM sockets would benefit from that as well.
 
> An event interface is much cleaner than monitoring. It still breaks
> the scalability abstractions, except for ROUTER where it's a good fit.
> 
> -Pieter

I agree that an application should not care about the topology and
such things.

But that assumes the proper protocol exists to handle such things
transparently for the application. For simple cases like REQ/REP or
PUB/SUB a protocol exists. But to build a more complex protocol on top
of basic sockets requires some interaction with the topology.

Maybe the right thing would be to implement PPP (and PPPP) directly in
zmq as new socket types. But I think that would be bad in the long
term. Adding a new socket type for every protocol would be a nightmare
to maintain I bet. Exposing events on the socket and implementing
protocols as a middle layer seems better.

My test setup for PPPP looks like this:

QT client <--inproc--> PPPP SUB helper <--tcp--> PPPP PUB helper <--> server

MfG
	Goswin