[zeromq-dev] Topics for ZMTP 3.0

[Pieter Hintjens]

On Mon, Jun 10, 2013 at 2:34 PM, Ciprian Dorin Craciun
<ciprian.craciun at gmail.com> wrote:

>     Indeed a rejection message should ease the debugging.

I'll look at adding this but I'm going to wait until we have more
running code so we can really test the error handling / reconnects.

>     I think this is the main "Achilles' heal" of ZeroMQ:  the
> "advised" (better said "required") controlled environment.  You can't
> use ZeroMQ "over the Internet" without special measures...

Over clear-text, right. The whole point of security is to make this
safer. There will still be DoS attacks until we harden libzmq
properly.

>     Then, if DTLS already seems a good enough solution, why not choose
> that as the specified solution?

There's no exclusive solution; anyone can provide new mechanisms. I'm
using NULL, PLAIN, and CURVE to keep the dependencies minimal and
because they look like a good fit. Also I want to use CURVE for my own
apps.

>     This solution still doesn't address my main concern, that of the
> security solution (via ZAP) needing to support meta-data handling,
> even though that meta-data doesn't refer to the security itself.
>
>     My proposal would be something like this:
>     * phase (1) establish channel, and negotiate ZMTP version (the handshake);
>     * phase (2) via commands, and delegated through ZAP, negotiate security;
>     * (after this security is applied;)
>     * phase (3) (currently missing and folded in phase (2)) negotiate
> any other connection related meta-data (like socket type, identity,
> etc.);
>     * phase (4) switch to message mode;

ZAP is not a delegate for security negotiation; it is just an
authentication bridge. At this stage that's all we need. I expect
we'll redesign the internals of how we implement mechanisms at least
once before it's stable.

>     I don't get this, mainly because I didn't follow closely enough
> ZeroMQ developments.  But isn't the identity of the remote peer chosen
> by that remote peer itself?  In this new version the identity is
> chosen by the local peer?
>
>     I would think that enforcing identity specification could at least
> help in debugging, manly because it's the only way in which you can
> determine the source of a message (if the source isn't embedded in the
> message by the application).

Sorry, you need to read up how ZeroMQ identities work (REQ, DEALER,
ROUTER talking to ROUTER sockets only). They're not a generic
mechanism. The name is poor; it's a reply routing handle.

The criteria for adding something to ZMTP is that it solves real
problems we face today with ZeroMQ applications. No speculative or
"could be useful" engineering, please.

>     It could be used by future ZMTP libraries to "augment" the
> protocol without actually inventing a new version of it.  (However as
> said the implementers should refrain on putting too much logic in
> these extensions.)

Speculation, dismissed.

>     A practical example of this:  imagine that commands with the a
> "mandatory" flag already existed in v1 of the protocol.  In that case
> "subscriptions" could have been pushed by the `SUB` sockets to `PUB`
> sockets as an "optional" command, without introducing a new type of
> socket, and without breaking the older libraries, because these
> commands would have been ignored.

Counsel, please approach the bench. You're leading the witness here
with pure speculation. Either you have hard evidence that this is
fact, or you're wasting the court's time.

>     I understand that there needs to be a balance between simplicity
> and functionality, and why ZMTP strives not to be AMQP.  However the
> main problem with ZMTP (v1) was its "extreme" simplicity, which we now
> pay by having a third completely incompatible protocol, whose
> versioning negotiation relies on sending half of well defined
> messages...

Perhaps. I'd have liked to have seen at least a version header in ZMTP
1.0 but we didn't have that luxury.

I'm not interested in extensibility unless you can point to real use
cases. The meta-data is extensible because already have N fields. The
mechanisms are extensible because we have N of them. The commands
likewise, fully extensible.

I'd estimate 50% of AMQP 0.9.x was engineered because we felt it might
be useful, but was never used, and added significant (toxic) technical
debt.

ZMTP 1.0's extreme simplicity made it possible to implement trivially,
which lots of people have done.

>     As such a little "extensibility" wouldn't hurt in the long term.

Complexity always hurts.

If you want to experiment with variations of ZMTP, the spec is GPL
licensed specifically so that you can fork it freely.