[zeromq-dev] RFC: PPPP - Paranoid Pirate Publishing Protocol

Goswin von Brederlow goswin-v-b at web.de
Mon Jun 30 15:48:24 CEST 2014


Hi,

I came up with an extension to the Paranoid Pirate Protocol [4] for
use in a PUB/SUB pattern. The protocol should work with ROUTER/DEALER
or XPUB/XSUB sockets I hope. With ROUTER/DEALER the publisher has to
manage sending messages to all peers itself, with XPUB/XSUB zmq will
handle that part. With ROUTER/DEALER the resends can be more
specifically targeted.

Questions:
- Would this work with epgm?
- Does XPUB/XSUB work with CURVE?

MfG
	Goswin

----------------------------------------------------------------------

PPPP - Paranoid Pirate Publishing Protocol 
##########################################

The Paranoid Pirate Publishing Protocol (PPPP) defines a reliable
publish-subscribe dialog between a publisher and subscribers. PPPP
covers publishing, subscribing, ACK/NACK of messages, heartbeating,
reconnecting and custom messages

    * Name: rfc.zeromq.org/spec:???/PPPP
    * Editor: Goswin von Brederlow <goswin-v-b at web.de>

License
=======

Copyright (c) 2011 iMatix Corporation.
Copyright (c) 2014 Goswin von Brederlow <goswin-v-b at web.de>

This Specification is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 3 of the
License, or (at your option) any later version.

This Specification is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, see <http://www.gnu.org/licenses>.

Change Process
==============

This Specification is a free and open standard[2] and is governed by
the Digital Standards Organization's Consensus-Oriented Specification
System (COSS)[3].

Language
========

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119[1].

Goals
=====

The Paranoid Pirate Publishing Protocol (PPPP) defines a reliable
publish-subscribe dialog between a publisher and subscribers. PPPP
covers publishing, subscribing, ACK/NACK of messages, heartbeating,
reconnecting and custom messages.

The goals of PPPP are to:

  * Allow the publisher to detect disconnection of a subscriber, through the use of heartbeating.
  * Allow a subscriber to detect disconnection of the publisher, through the use of heartbeating.
  * Allow subscribers to detect message dropouts, through sequence numbers.
  * Allow the publisher to know messages were delivered, through ACK/NACK feedback.
  * Allow custom messages from subscribers to the publisher.

Architecture
============

Roles
-----

PPPP defines two types of peer:

    * A "publisher" sends messages to "subscribers" and receives
      ACK/NACK, subscriptions and custom messages.

    * A "subscriber" subscribes at the start and listens for messages
      from the "publisher" and sends ACK/NACK feedback. It can also
      send custom messages to the "publisher".

Overall Conversation
====================

PPPP connects a single publisher and a pool of subscribers. We do not
specify which peers connect to which, but usually subscribers will
connect to the publisher.

A PPPP conversation consists of two intermingled dialogs, as follows
('P' represents the publisher, 'S' the subscriber):

    Synchronous dialog:             Asynchronous dialogs:
    ---------------------           ---------------------
    S: SUBSCRIBE                    Repeat:
    P: SYNC (optional)                  S: HEARTBEAT
    Repeat:                         Repeat:
        P: PUBLISH                      P: HEARTBEAT
        S: ACK/NACK (periodically)  Ocasionally:
                                        S: CUSTOM
                                        P: PUBLISH (optional)

Breaking this down:

    * The subscriber initiates the conversation by subscribing.
    * The publisher syncs state with the subsriber.
    * The publisher publishes messages.
    * The subscriber replies with an ACK/NACK, and this repeats indefinitely.
    * The subscriber sends HEARTBEAT at regular intervals to the publisher.
    * The publisher sends HEARTBEAT at regular intervals to the subscriber.
    * The subscriber can send CUSTOM messages to the publisher.
      The publisher may or may not publish a reply.

The first message in any conversation MUST be S:SUBSCRIBE.

Command Specifications
======================

SUBSCRIBE
    Consists of a 1-part message containing the single byte 0x01
    followed by the channel to subsribe to. A subscriber can
    subscribe to multiple channels.
SYNC
    Consists of either a full state dump or a backlog of messages. Wether
    full state dump or backlog is used (or neither) is application specific.
PUBLISH
    Consists of a multipart message containing a channel as first frame,
    a channel specific sequenze number and paylod in additional frames.
ACK/NACK
    Consists of a multipart message containing the single byte 0x02
    followed by the channel as first frame, a second frame containing
    the highest sequenze number received (ACK) and optionally frames
    containing ranges of sequenze numbers that were not received (NACK).
HEARTBEAT
    Consists of a 1-part message containing the single byte 0x03.
CUSTOM
    Consists of a multipart message containing the single byte 0x04
    as first frame followed by paylod in additional frames. The
    format of the payload is application specific.

ACK/NACK Specification
======================

Every published message has a channel specific sequenze number. The
publisher MUST preserve each published message until it is
acknowledged by all subscribers that are not permanently disconnected
(see Heartbeating below). When an ACK/NACK message contains NACKs the
publisher MUST resent the listed messages again.

Not every message needs to be ACK/NACKed individually, allowing a
single ACK/NACK to cover multiple messages. But an ACK/NACK MUST be
send by the subscriber within an agreed-upon first idle time.
Collectively a subscriber MUST ACK/NACK every message, even
duplicates. It is application defined wether messages are delivered to
the application in sequenze order. But the application MUST receive
every message exactly once.

If no ACK/NACK is received from a subscriber after some multiple of
the first idle time (usualy 3-5) and no message was send to the
subscribe in that time then a message, preferably the last, for that
subscriber should be resend. The message to resend can be picked to
cover the largest number of subscribers. In such a case an ACK/NACK
reply from a subscriber implicitly NACKs the range of messages between
the message send and the last message. The idea here is to only resend
a small amount of data when a subscriber dies.

Heartbeating Specification
==========================

Heartbeats are send after an agreed-upon idle time. As a special case
the first heartbeat is send after a shorter agreed-upon first idle
time (see ACK/NACK). This is to allow faster detection of message loss
of the last message while keeping traffic during idle times low. Any
other traffic counts as implicit heartbeat. A peer can consider the
other peer "disconnected" if no heartbeat arrives within some multiple
of that interval (usually 3-5).

If the subscriber detects that the publisher has disconnected, it MUST
send SUBSCRIBE again to start a new conversation.

If the publisher detects that the subscriber has disconnected, it
SHOULD stop sending it messages of any type. Depending on the type of
SYNC used pending and future messages must be preserved though. The
backlog of messages can be droped after a grace period, after which a
subscriber is considere permanently disconnected. If a subscriber
reconnects after that grace period a full state dump must be used for
SYNC.

References
==========

Bibliography
------------

1. "Key words for use in RFCs to Indicate Requirement Levels" - ietf.org
2. "Definition of a Free and Open Standard" - digistan.org
3. "Consensus Oriented Specification System" - digistan.org
4. "PPP" - http://rfc.zeromq.org/spec:6



More information about the zeromq-dev mailing list