[zeromq-dev] Notes from a hackathon

sven.koebnick at t-online.de sven.koebnick at t-online.de
Mon Feb 16 10:34:38 CET 2015


Hi *! 

I just read through this thread and would like to share some
thoughts about several things in discussion: 

I picked ZMQ to have a
straight forward lib that is easy to handle and makes it easy to both
stick to high level things (application concerned) as well as going into
details where needed (adressing, routing etc.). 

Of course, it's
discussable to have routing data somehow "mixed" with application data
(adress being just another frame >>next to<< the net load and thus
having the need to "find" the real data between other things). But
overall it is a slender approach that works well. 

I apprechiate the
idea of improving the separation between technical data and semantic
data by (as I understand it) replacing technical frames (routing etc.)
by metadata structures, but what I'm really afraid of is the fealing,
that you are going to blow up every logic that currently deals with ZMQ3
and 4, replacing things by a new API completely incompatible in it's
logical structures with old APIs. 

I had (and still have) heavy work
with migrating from the antique ZMQ2 to actual ZMQ4 lib, just because
the API allready switched in some integral parts. E.g. it's a constant
question in background, if the used function will take ownership and
free up data after sending or if there will be a (new) leak in my code
(using C++). 

I wrote a (rather large) wrapper arround every network
logic just to be able to easily deal with another API switch. E.g. in
ZMQ2 I could fetch real data and adressing in separate reliable
functions in zmq itself. In ZMQ4 I have to tell adress and data appart
by myself. Easy and short code will break as soon as the routing changes
somehow. Flexible code will be complicated and timeconsuming and I
intensionally took ZMQ because it does some valuable things for me (e.g.
getting routing frames separated from application data). 

What I did
not really like, was the abstraction of ZMQ wrappers in the form of
void* everywhere. This increases the issues concerning type safety
(compiler wont complain when accidencially having an missing * or an
ampersand too much). 

For being able to deal with application data
(high level) as well as with technical data (routing to several workers
etc.) I had to use different wrapper APIs (native libzmq as well as
CZMQ, but CZMQ has "hidden" internal structures and I did not find ways
to take classes from CZMQ and work on them using libzmq (e.g. because of
that hiding void* _ in zmsg_t). 

Indeed, I was thinking about investing
even more time in my own wrapper (having funcs like address(int),
data(), size(), routing_path_length() and others) to make it even more
foolproove, but after having read this thread, I'm afraid, that I'm just
about to do a thing, that you are also just planing, expecting, that you
will do it in another way and my wrapper breaks again. 

Long talk -
short sense: IMO it's the most valuable thing for every framework,
library and whatever to have a STABLE API, that users can rely upon to
work for several versions. 

Call it queer, but I expect a library of
such a high quality (as zmq definitely is) to be API-stable, requiring
just to link the new lib, do a test and be productive again. 

Why do I
write all this? 

As I wrote, I'm still busy in switching from a
(working) ZMQ2 to ZMQ4, which somehow does not work stable in our
system. Propably, this is NO bug in zmq, but the changed API from 2 to 4
required such a big deal of changes, that I now cannot find my bugs. The
network stuff was developed and tested long ago having a rather slender
skeleton. Now, there are many thousand lines (5 digits!) of code, I
allready did a complete leak checking etc. inside this complex system
and still it does loose messages (upon MY bugs surely). 

Writing a
minimal test case does not show up the message loss. 

In short again:
switching the API required too much chances in a complex system, so when
the API changes again even in its logic, I'll trhink about going to
another MQ system and rewrite the whole wrapper again, but hoping it's
the final. 

... but I dont want to ! 

ZMQ is great, but please do not
break application code interface again! 

Last thing: I did not yet get
the ides of making socket thread safe. Your whole concept was build atop
the statement: "we don't need thread safety, because we separate via
sockets: each thread it's own socket, so no thread battles". And I think
you were right with that. Why now change this for such a high price
(breaking the API logic)? 



Am 2015-02-09 10:02, schrieb
Pieter Hintjens: 

> James, thanks a lot for this report. It's the kind
of data we don't
> get often enough.
> For me it's become clear over
the last years that very few people can
> properly write messaging
layers. Actually I've known for decades that
> only a small slice of
programmers are fit to write platform code.
> ZeroMQ promised to
democratize this, yet it can't solve human
> limitations. OTOH we've
taught thousands of people to write brokers
> (quite amazing, really).
On the other hand, we've probably enabled
> billions of lines of horrid
> My answer is to provide more pre-packaged pieces like Zyre
> Malamute, and to slowly steer people away from writing their own
brokers and stacks. And if they do, to be less creative about it, and
better equipped with tools like zproto that do the work for them. (And
yes, people then abuse zproto, and we need more answers for that...)

> -Pieter
> On Mon, Feb 9, 2015 at 1:57 AM, James Gatannah
<james.gatannah at gmail.com> wrote:
>> This ties into the heart of some
related discussions we've been having at work recently. We have a bunch
of new hires who are trying to come to grips with a messaging protocol
that's evolved over the past year or so as we've gotten more familiar
with 0mq and adapted it to our changing needs. It's really just
meandering commentary about the way I perceive the proposal as it
relates to the way we're currently using the library (in C++, C#,
python, and clojure via jzmq). It may be a complete waste of time, or it
may be a validation of the proposed changes. Our current incarnation
looks something like: Frame 1: address header (originally for pub/sub)
Frame 2: bundle of serialized data (with a bunch of HTTP-style metadata,
like Verb, URI, headers, the "actual" message body etc). Normal
router/dealer identity frame sequences. New team members would very much
like to split that apart into 7 or 8 more frames, just so the "actual"
message body is totally isolated. We could probably accomplish something
similar using zproto, though I'd much rather just cram everything into
one single frame. We can't do either, mainly due to time constraints
around changing the C++ layer. Pieter Hintjens <ph at imatix.com> wrote:

>>> I think we can either get the routing id as a message property,
and/or add a "reply to message" semantic that does this implicitly. The
second is easier for simple servers, the first is better for realistic
servers. Since it only applies to server sockets, I think we would
benefit from a zserver_reply () method (or similar).
>> For our app, we
have "clients" that have to establish the initial connection (they use
dealer sockets. The central messaging hub has the router). After that,
those "clients" really switch to being "servers." We're attaching the
identity frame(s) to each incoming message so we can use them to send
the response. We're also tracking them on the "server" side when a
"client" first connects so the "server" (which will usually act more
like a client) can send off its requests. We decided on Friday to just
start to thinking of everything as a "peer" and just think in terms of
"message events" instead of thinking in terms of "client/server" and
"request/response". We haven't run into scenarios where peers drop
connections, but we're also barely at the PoC stage. Arnaud Kapp
<kapp.arno at gmail.com> wrote: 
>>> I think thread-safe socket would
be cool. As in great to have in some particular situation, but most of
the case it shouldn't be needed.
>> This would be a huge win for us. We
have a ridiculously complicated pair of event loops in our central
message hub(s) to make the multi-threaded/asynchronous pieces work
correctly. I wrote it, and I'm absolutely positive that I'll regret it
in a couple of months. That's on the clojure side, where multi-threading
is about as simple as it can get. We can get away with single-threading
in the C++ parts. The .NET parts are going to be really ugly. The python
pieces...well, I keep being forced to say "I told you so" about them.

> _______________________________________________
> zeromq-dev mailing
> zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev [1]

[1] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150216/afd3bc6e/attachment.htm>

More information about the zeromq-dev mailing list