[zeromq-dev] What differentiates connected peers on a ZMTP point of view ?

Laurent Alebarde l.alebarde at free.fr
Thu Feb 13 17:37:49 CET 2014

I have finally managed to proxy ZMTP over a ZMQ_STREAM socket tunel, 
with several clients and several workers, with the following architecture:

_______________client A___________________ 
proxy                  __worker B __
                     _________tunel A______ _____midpoint_____
Client  ----tcp---- frontend   /   backend ----tcp---- frontend / 
backend ---tcp---- Worker
DEALER              ZMQ_STREAM      DEALER             ROUTER 
CURVE                               NULL NULL                          CURVE

I have to admit I have had a very hard time to achieve this.

Thanks again Goswin.


Le 11/02/2014 10:02, Goswin von Brederlow a écrit :
> On Mon, Feb 10, 2014 at 05:31:34PM +0100, Laurent Alebarde wrote:
>> Le 10/02/2014 10:43, Goswin von Brederlow a écrit :
>>> On Thu, Feb 06, 2014 at 04:34:03PM +0100, Laurent Alebarde wrote:
>>>> Hi Devs,
>>>> I wonder please what differentiates connected peers on a ZMTP point
>>>> of view ?
>>>> To be more clear, let's consider this network:
>>>> Client 1
>>>> ---------------------------------------------------- Server
>>>> Client 2 --------------------------------------------------/
>>>> In ZMTP internals, I assume it creates a pipe associated with each
>>>> origin address, and on an API side, it provides one identity
>>>> associated to each one. So it creates one mechanism for each pipe
>>>> and knows which instance to use every time in the stateful handcheck
>>>> process. Is it correct ? there is a bijective association between
>>>> the origin address and the identity ?
>>>> Let's say that now we have a proxy in between:
>>>> Client 1 ----------------------- Proxy
>>>> ----------------------- Server
>>>> Client 2 ------------------/92.123.321.22
>>>> ZMTP on server side has no way to differentiate between Client1 and
>>>> 2. Every message arrives in the same pipe which is the one
>>>> corresponding to its closest peer: the proxy and its address. The
>>>> only way for the server to differentiate the clients is at
>>>> application level, when identities are stacked in the former frames
>>>> of each message ?
>>>> I am a bit confused. Can someone clarify for me please and point in
>>>> the libzmq code what is taken into account to identify the pipes ?
>>> http://zguide.zeromq.org/page:all#The-Extended-Reply-Envelope
>>> A zmq message does not have just one identity. It has any number of
>>> identity frames followed by an empty delimiter frame followed by data
>>> frames. Normaly on each hop an identity frame is added. Depending on
>>> the socket type when sending the first identity is used to decied
>>> where to send the message and on recieve the identity of the sender is
>>> added to show where it came from:
>>> Client 1 sends:    '' 'request'                      # REQ adds ''
>>> Proxy recieves:    <client 1> '' 'request'           # ROUTER adds <id>
>>> Proxy sends:       <client 1> '' 'request'           # DEALER
>>> Server recieves:   <proxy> <client 1> '' 'request'   # REP handles <id>s
>>> Server replies     <proxy> <client 1> '' 'reply'     # REP handles <id>s
>>> Proxy recieves:    <client1> '' 'reply'              # DEALER
>>> Proxy sends:       <client1> '' 'reply'              # ROUTER strips <id>
>>> Client 1 recieves: '' 'reply'                        # REQ strips ''
>>> The REP/REQ sockets handle the identity frames for you so the
>>> appliction never sees them. ROUTER adds/strips the first identity
>>> automatically while DEALER lets you see them all.
>> Thanks Goswin,
>> My question here is on the internals of libzmq: how it manages
>> identities ws pipes.
>> Continued below.
>>>> So, if my above hypothesis are right, as I want to manage to proxy a
>>>> ZMTP mechanism like in the following:
>>>> _______________client A___________________
>>>> ____________________server_________________
>>>> proxy                  __worker B __
>>>>                      _________tunel A______ _____midpoint_____
>>>> Client  --inproc--- frontend   /   backend ----tcp---- frontend /
>>>> backend --inproc-- Worker
>>>> CURVE |              CURVE
>>>> |
>>>> I have to create a pool of sockets in the proxy backend, say 1,000
>>>> if I want to authorize 1,000 simultaneous connexions, and dispatch
>>>> the clients to a sticky socket, and one and only one client per
>>>> socket.
>>>> \|/
>>>> Client 1 ----------------------------------------------> socket 1:
>>>> port 10001
>>>> Client 2 ----------------------------------------------> socket 2:
>>>> port 10002
>>>> Client 1000 ----------------------------------------------> socket
>>>> 1000: port 11000
>>>> So, the worker is binded to all these ports. Still, does it work
>>>> here ?  Does ZMTP creates a pipe per address on the worker side,
>>>> even if they are binded to the same socket (the answer looks obvious
>>>> but I prefer a confirmation) ?
>>>> Cheers,
>>>> Laurent
>>> No. You use one socket for everything and add the identity of the
>>> client to the message as you proxy it. Then on the reply you get the
>>> identity of the client back so you know where to send it. Since ROUTER
>>> adds the identity implicitly all a proxy has to do is real all message
>>> frames from the ROUTER and write them to the DEALER and vice versa.
>>> zmq does the rest as if it where magic.
>> The worker being set with CURVE, the "classic" backtrack you
>> describe cannot work. I cannot attach a backtrack when I send a
>> message from a proxy to a CURVE worker.
>>> MfG
>>> 	Goswin
> Sure you can. The CURVE encryption is irelevant and transparent to the
> design. What you have is a simple case of tunneling:
> Client ---- Tunnel A -- T1 -- T2 -- T3 -- Tunnel B ---- Server
> You have 2 routing chains here. One is the tunneling itself:
> Tunnel A -- T1 -- T2 -- T3 -- Tunnel B
> Messages send from Tunnel A should arrive Tunnel B with the IDs of
> Tunnel A, T1, T2 and T3 in the message so the reply goes back to
> Tunnel A. Although if the tunnel is strickly a 1-1 connection you can
> skip the IDs since there is always only one way a message can go.
> The other routing is between Client and Server. If you use STREAM
> sockets then for them the tunnel is transparent.
> Client ----<Tunnel>---- Server
> So messages send by Client arrive at Server with the ID of Client in
> the message, which will be associated with the Tunnel B endpoint.
> Replies send to the client will go to the Tunnel B socket and get
> forwarded to Client with an ID of Server, associated with the Tunnel A
> endpoint.
> That is if you use STREAM sockets to tunnel the raw traffic. The
> identity the client and server sockets publish will get tunneled and
> associated with the tunnel endpoints. Or you get random IDs for them.
> For the application it will be the IDs of Client and Server directly.
> MfG
> 	Goswin
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140213/e606c227/attachment.htm>

More information about the zeromq-dev mailing list