[zeromq-dev] San Francisco meetup today!

Maxime Petazzoni maxime.petazzoni at bulix.org
Wed Mar 9 08:56:18 CET 2011


Hi all 0MQ fans,

As you most likely know we had tonight the first (ever?) ZeroMQ meetup
in San Francisco, kindly hosted by Loggly at 78 1st St in the heart of
the city.

We had a very nice and friendly crowd there of about a dozen people all
using, or interested by, in some way, ZeroMQ, starting by Loggly
themselves.

Our discussions were centered around what we felt would be nice-to-have
features in the mid-to-long term. Martin hinted at a few things
currently (or soon-to-be) in the pipeline as well, and it sort of went
from there.

One topic we discussed was how we should handle changes that break
compatibility of the wire protocol used by ZeroMQ. Given the size of the
community, it was agreed that if this should happen, it would better be
done sooner rather than later, as a new major revision (and thus most
likely in a new repository like zeromq3), in an extensible way and with
a long-enough period of warning before the change. The rationale behind
this discussion is that some of the desired features (see below) like
port multiplexing require some enhancements to the headers sent by
ZeroMQ.

Amongst the proposed functionality were:

  * graceful validation of a connection, i.e. have a clearly defined
    behavior when someone tries to connect to incompatible socket types
    together. This would be achieved by transmitting the socket type on
    the wire at connection time, with some check on the server side to
    validate this is an acceptable mix and match;

  * port multiplexing, which would allow sharing of a single listening
    port for multiple sockets, using the concept of "sub ports". This
    would obviously only be possible within the same ZMQ context
    (because ultimately only one bind(port) can be requested from the
    kernel for a given port number), but would allow someone to attach
    several sockets to this one listening port, and to connect to
    different sockets on one remote port.

    You could for example say:

      zmq_bind(socket1, "*:5555.0");
      zmq_bind(socket2, "*:5555.1");

    To bind these two sockets to the same listening port, but with
    different subports. On the client side, you can connect to either
    of them with:

       zmq_connect(socket, "server:5555.0");

    If no subport is specified, bind() would default to all subports,
    and connect would default to subport 0;

  * integrate handling of timeouts, re-sends w/ de-dup and eventually
    checkpointing in REP/REQ sockets, directly as base socket features
    handled by ZMQ.

    Timeouts would make using REP/REQ sockets easier so you don't get
    stuck in the recv() state if you never get an answer.

    Re-sends would make ZMQ automatically, at periodic and eventually
    configurable intervals, resend the last message until you get an
    answer (useful if the other end disappears after receiving the
    message but before sending a reply). De-duping (on the receiving
    end) is just a required feature in this case where you're still
    alive but haven't answered *yet*.

    Checkpointing would allow ZMQ to send "backchannel" information
    every N messages to inform the sender you've successfully reached
    a "checkpoint", and that if the server ever needs to resend data
    it can do so starting from the last known checkpoint.


Note that this is a very, very rough summary of what we discussed, and
may not be entirely accurate -- and for that I apologize. Feel free to
comment, correct, criticize and contribute to these ideas! ZeroMQ can
only become better for us users iff we express our needs somehow.

Martin also reported that he applied on behalf of ZeroMQ as a Google
Summer of Code organization, and apparently discussion already started
on that particular subject on this very list. Having SoC students work
on ZMQ would be a great thing IMHO, so we should make sure we have some
project ideas (that should be fine) and that we'll welcome candidates
with open arms :)

Finally, we briefly talked about automated testing, in particular for
long-lasting "burn tests" scenarios that would help test core changes to
ZeroMQ both at the C API level and at the language bindings level. Feel
free to propose/submit test scenarios and/or their implementations!


And that's it for tonight! It was a great meetup, thanks again to Loggly
for hosting, and we hope to see more of you next time!

- Maxime
-- 
Maxime Petazzoni <http://www.bulix.org>
 ``One by one, the penguins took away my sanity.''
Writing software in California
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110308/105298c1/attachment.sig>


More information about the zeromq-dev mailing list