[zeromq-dev] Zyre/curve and peer identity/access

Justin Azoff justin.azoff at gmail.com
Fri Jul 21 21:21:58 CEST 2017


See below for an update on zyre/curve progress, the short version is
that it works.

Now that we can run a curve enabled mesh, I'm not sure how message
authentication should be handled.  We can ensure that only clients we
allow are able to connect to us, but how can we prevent one client
from impersonating another?

I've submitted a PR for pushing the User-Id out of ZAP and into the
socket metadata


With this in place zre_msg_recv will have access to which public key
sent the message, but I'm not sure how that should be presented back
up to the zyre user without changing the 'outbox' API.  HELLO has a
headers field, but the other messages do not.

I can add the peers identity to a header and pass it up to the
application as part of the HELLO and then save that into a hash as a
mapping from uuid=identity, but it wasn't clear to me if a malicious
peer could spoof their UUID on later messages.

For what we are doing, the name or uuid of the sender is irrelevant,
we want to know which key was used to transmit the data.  In practice
the name of the sender would likely match the name metadata in the

Another thing I'm not sure how to implement without totally forking
zyre is access control via cert metadata.  Imagine 2 certs like this:

    name = "Restricted client"
    read-groups = "public,test"
    write-groups = ""

.. and some other cert

    name = "trusted client"
    read-groups = "*"
    write-groups = "sensitive,public,test"

Where the first client is allowed to join public and test groups but
not allowed to transmit data.
The 2nd client can join any group and send to 3 groups.

For handing incoming SHOUT messages we could solve this in the
application: if we receive a SHOUT from a peer that does not have
access to a group, drop the message.  This doesn't work for outgoing
SHOUT messages though. We would need to hook into
zyre_node_join_peer_group and prevent a peer from joining a group in
the first place.

Maybe zyre needs a 2nd level of a zauth/zap like protocol for access control?

I have also been thinking a lot about how to handle long lived clients
and key revocation.  We also want to ensure that if a key is removed
from the store existing sessions are stopped.

With the User-Id metadata we can now do

        zframe_t *frame = zframe_recv(server);
        const char *user_id = zframe_meta(frame, "User-Id");
        if(user_id!=NULL) {
            zcert_t *c = zcertstore_lookup(store, user_id);
                //Certificate no longer exists in store!

but I kinda wish there was a way to do

zstr_sendx (auth, "REAUTH INTERVAL", 3600, NULL);

to make ZAP re-authenticate the connection and drop the connection if
the client is no longer authorized.  I have a feeling this is not
easily done with the current design.

zyre/curve progress

I have made a lot of progress on curve enabling zyre.  I merged Wes's
branch and mine together and have something that is good enough for
testing.  I did run into a really crazy bug where self->inbox was
somehow dropping the zsock_set_curve_server option.  It works as long
as I start the node first and then setup curve.

The branch is here:


It looks a little longer than it should because there's an example in
there and I had to re-order some functions to make the compiler happy.

It's working pretty well, I have 5 nodes on the internet, 2 of them
behind NAT (with port forwarded), joined into a mesh.

There's some minor non-curve things that can be improved related to
how HELLO and discovery work if you are behind NAT.  There are also
issues with dual homed or dual stack v4/v6 hosts.

I think a lot of the fixes will simply involve using the
'Peer-Address' metadata and letting nodes just automatically connect
back to whatever IP address connected to them first.  I 'solved' this
in my simpledisco discovery service by doing:

    const char *peer_address = zframe_meta(command_frame, "Peer-Address");
        if(key && strlen(key) > 8 && key[6] == '*') {
            char *new_key = zsys_sprintf("tcp://%s%s", peer_address, &key[7]);
            zsys_debug("zsimpledisco: Rewrote %s to %s", key, new_key);

Which rewrites tcp://*:port to tcp://peer-address:port automatically.
It still feels a little hackish, but it works perfectly for peers
behind NAT.

- Justin

More information about the zeromq-dev mailing list