[zeromq-dev] Notes from a hackathon

[Pieter Hintjens]

Thanks for the questions. I'd try to flatter by saying how great these
questions are, except I know you're all way too smart to fall for
that. :)

Let me answer a bunch of questions in one go:

* Sorry for not posting this here. I've written a more detailed
analysis of the motivation for this thread (and experiments):
http://hintjens.com/blog:84.

* The main goal with this is simplicity and accuracy. I'm tired of
teaching people ZeroMQ constructs that are clearly over-designed and
disconnected from reality in subtle ways. Each apology for friction
("sorry, but don't use REQ-REP, they are kind of weird", "sorry,
IDENTITY is not really identity", "sorry, message sometimes means
frame and sometimes message") really signals costs up and down the
stacks.

* Sessions require protocol commands that are invisible to "normal"
use. That means either protocol support (e.g. command frames) or some
layering on top of ZeroMQ blobs.

* Picture sending solves a cross-language problem, namely how to
construct zproto-style commands without using dedicated codecs. While
the API is C-ish, the functionality is not. I'd expect that to be
wrapped in bindings.

* Per-language serialization is an anti-pattern. It's common, and fine
as such, yet ZeroMQ should IMO always strive to be cross-language.

* REQ-REP between threads may make sense. We certainly use it (ZAP,
for authentication).

* Is this discussion C4 compatible? Yes, no, this is chatter, not a
plan or roadmap. Until pull requests hit the road, it's all vapour.
However the problems I restated in my blog article are real, to many
people, and should be aired.

* Multipart does easy serialization, yes. Not in C though. We've
learned how to do variable list serialization with zproto-compatible
picture sending in CZMQ. Is this a sufficient alternative? I think so.
zsock_send() was essential to making actors cheap enough to use
widely. egrep zsock_send *.c in malamute-core/src.

* Locking on socketsL: we did in fact have a pull request for this
once. It was... nasty. We reverted it. We breathed a sigh of relief.
However looking back, the problem was valid and has never been solved
since.

* If you're doing multipart a lot in an application, you can replace
that with zproto. You already have contracts then, why not make them
explicit and documented? That's always a win in my experience.

* Multi-hop request reply is a strange beast and I'm keen to
understand those architectures that use it. My suspicion is that we've
shifted our vision of what ZeroMQ "is" over the years. My own
experience seems plausible, yet I distrust it. So, the stories in my
article are please for falsification, not attempts to convince.

* I think we have learned to not break existing code. It's going to be
interesting to run multiple models at once. We do this in CZMQ and it
works OK. There isn't much overhead.

* Authentication and routing information should be (MUST be) two
totally separate layers. CURVE certainly does not use the routing ID
(even if it's still misnamed as "IDENTITY"). You can have the same
client on N connections (obviously). You authenticate a client using a
mechanism-specific key: PLAIN user id, or CURVE public key.

-Pieter