[zeromq-dev] new API proposal: zmq_metasocket - your thoughts ?

Laurent Alebarde l.alebarde at free.fr
Mon Jan 20 11:23:16 CET 2014

Hi Devs,

 From the exchange below, I wish to develop a complementary API for 
libzmq. For the ones who have not followed the thread below, it would 
enable to encapsulate ZMPT in a lower protocol, itself embedded in 
another ZMTP layer (tunel backend). My goal is to be able to proxy CURVE 
and so its handcheck end-to-end, but probably many other use cases are 

Use case:

void *client = zmq_metasocket (ctx, ZMQ_DEALER, ZMQ_DEALER);

What it does:

|_______________client ____________________
                     _________tunel _______
Client  --inproc---|||frontend|    /   backend
DEALER              ZMQ_STREAM|||     DEALER|  

Normal API will adress the socket on the left. Everything else is 
hidden. The second ZMQ_DEALER is for the tunel backend. It could be 
ROUTER also, and possibly anything else.

The tunel will use a transport: "inproc://tunel00x", with x static and 
incremented for each new metasocket.
The internal socket will be able to be configured with: zmq_setmetasockopt
With setmetasockopt, it will be possible to plug some utility functions 
between the tunel frontend and backend.

Any suggestions or contributions welcome.



Le 18/01/2014 16:08, Laurent Alebarde a écrit :
> Le 18/01/2014 11:58, Goswin von Brederlow a écrit :
>> On Fri, Jan 17, 2014 at 11:40:13AM +0100, Laurent Alebarde wrote:
>>> Hi Devs,
>>> I am doing some experiments (cf at the end for the goal) and wonder
>>> where I could hook the beginning of a message both reading and
>>> writing.
>>> What I want to do is prepend an outgoing message and trim an
>>> incoming message (for any mechanism looks simpler) when some option
>>> is set.
>>> Typically, when the option is set, when a new message is received, I
>>> want to be able to store a certain amount of the first bytes in a
>>> new mechanism buffer, and skip them. These bytes are experimental
>>> and not ZMTP, so I have to skip or trim them to not hurt ZMTP. When
>>> a message is sent, I want to initialise the buffer with a copy of
>>> what I previously stored before ZMTP write the buffer. If it is
>>> outpos, I will do outpos += store_size;
>>> So, the question is where & when I should do that to not hurt ZMTP.
>>> FYI, The aim of this experiment is to manage to achieve CURVE
>>> proxying. My investigations and tests so far lead me to prepend the
>>> client ID in every message sent to the CURVE worker, and that the
>>> worker send back this information.
>>> Cheers,
>>> Laurent
>> How about this? (thread names in "")
>> "A"        REQ    <-inproc-> STREAM "Tunnel A"
>> "Tunnel A" DEALER <---tcp--> ROUTER "Midpoint"
>> "Midpoint" DEALER <---tcp--> ROUTER "Tunnel B"
>> "Tunnel B" STREAM <-inproc-> REP    "B"
>> The "Tunnel A" and "Tunnel B" use a STREAM socket to get at the
>> on-wire data. Each data packet send they envelope into a ZMQ message
>> and send out their other socket. As the name suggests this tunnels
>> data from "A" to "B" without the "Midpoint" parsing the data. Now
>> the data can be anything, e.g. a CURVE encrypted and authenticated ZMQ
>> connection.
>> Does that help?
> If I catch it, my initial architecture attempt:
> |                  ___________________server__________________
>                   ____________proxy___________
> Client  ------- frontend       /      backend ------- Worker
> DEALER          ZMQ_STREAM         ZMQ_STREAM         DEALER
> CURVE                                                 CURVE|
> becomes:
> |_______________client A___________________             ____________________________server________________________________
>                                                                proxy                  ________________worker B ___________
>                      _________tunel A______             ______midpoint____            _____tunel B______
> Client  --inproc---|||frontend|    /   backend ----tcp---- frontend / backend --inproc-- frontend / backend --inproc-- Worker
> DEALER              ZMQ_STREAM|||     DEALER|              ROUTER     ROUTER             ROUTER  ZMQ_STREAM            DEALER
> CURVE                                                                                                              CURVE|
> which means I envelop ZMTP into whatever protocol I want, itself 
> envelopped again into ZMTP, otherwise said, I put my custom protocol 
> between two ZMTP layers ?
> As I have adapted your suggestion to my need, I should be able to 
> simplify to:
> |_______________client A___________________             ____________________server_________________
>                                                                proxy                  __worker B __
>                      _________tunel A______             _____midpoint_____
> Client  --inproc---|||frontend|    /   backend ----tcp---- frontend / backend --inproc-- Worker
> DEALER              ZMQ_STREAM|||     DEALER|              ROUTER  ZMQ_STREAM            DEALER
> CURVE                                                                                CURVE|
> Thanks very much Goswin ! Very smart solution. I have to diggest it a 
> little.
> Cheers,
> Laurent
>> MfG
>> 	Goswin
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140120/3180e3f4/attachment.htm>

More information about the zeromq-dev mailing list