[zeromq-dev] Notes from a hackathon

[André Caron]

Hi all,

Even if the topic seems rather closed, I'd like to contribute a few things
:-)

There is a lot of discussion about thread safety of sockets from the
sender's perspective.  The reader's perspective is even more interesting
IMO.  For example, if I zmq_poll() to wait for input, then zmq_recv() on a
socket and multiple threads do this, I doubt you can make that scenario
thread safe.  Not that I recommend that specific design to anybody, but I
think it's worth thinking more thoroughly of what it really means to
promise "thread-safe" ZMQ sockets because I doubt it's simply limited to
atomic zmq_send and zmq_recv for multi-part messages.

As for this talk about the recurring overhead of explaining to people that
"identity" is actually routing info and that "req-req" are weird etc., I
would also consider the cost of invalidating all the existing
documentation: ZMQ guide (printed versions), bindings API references and
dozens of blog postings and tutorial which are not properly tagged with
dates or ZMQ versions, some of which we may not be able to correct.  Take
the example of Python 3 which performed the rather simple change of
changing "print" to make it a function: they broke their hello world
example, creating instant distrust of both the language and the very widely
available reference books from new users.  I would advise to weight the
cost of having to deal with the new confusion against that of living with
existing concepts.

Also, I +1 the need for convenient multi-hop support.  Which I'm also
finding very, very useful in a project I'm currently working on.  Think of
having multiple Majordomo brokers in front of multiple applications: the
request received by the application can neatly carry a double reply
address, avoiding the need for the broker to carry any state about
individual requests.

I think this discussion was very interesting, especially if it leads to
some newer, more convenient APIs, but I would really appreciate if there
was some experimentation with some new socket types before the core ZMQ
concepts are revised :-)

Cheers,

André

On Sat, Feb 7, 2015 at 3:18 AM, Pieter Hintjens <ph at imatix.com> wrote:

> On Fri, Feb 6, 2015 at 9:41 PM, AJ Lewis <AJ.Lewis at quantum.com> wrote:
>
> > This doesn't address the use of having messages that don't need to be
> > decoded at each hop...
>
> Let me collect what I see as valid requirements here, from the entire
> thread:
>
> - a routing stack for multi-hop request-reply, which can be used
> without decoding the message
> - replacement of "identity" concept with something like "reply handle"
> - a simple message blob that reduces upstream costs for message handling
> - atomic send/recv methods that can potentially be made threadsafe
> - scatter-gather send/recv that allows zero-copy for efficiency
> - new higher-level client/server semantics that replace
> req-router-dealer-rep
>
> What we do not need to do:
>
> - move any zproto serialization into libzmq; this can happily live at
> higher layers
> - break or change any existing contracts, as everything works and is stable
>
> These can all be satisfied, IMO, with some new protocol designs and
> new client/server sockets, and new API classes. And we can do this
> slowly, over time, perhaps with faster experimentation in stacks like
> NetMQ that are more fungible.
>
> -Pieter
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150207/798f9ad1/attachment.htm>