[zeromq-dev] Can zmq_connect/zmq_disconnect be invoked on a socket while zmq_proxy_steerable() is running in a separate thread for the same socket?

Luca Boccassi luca.boccassi at gmail.com
Thu Oct 18 19:40:43 CEST 2018


On Thu, 2018-10-18 at 17:07 +0100, Luca Boccassi wrote:
> On Thu, 2018-10-18 at 15:05 +0000, Martin.Belanger at dell.com wrote:
> > I'm designing a multi-node pub-sub pattern (each node is a separate
> > computer running Debian Linux). The number of nodes is configurable
> > and can change over time (i.e. through configuration I can add or
> > remove nodes). I want to use XPUB-XSUB proxies on each node as
> > depicted below. The following link is for a picture showing the
> > whole
> > topology: https://docs.google.com/drawings/d/1WZ-h_3cP1wc0WEEBmtrhH
> > 6C
> > DNakyNE4KTb-GApIV7Pk/edit?usp=sharing
> > 
> > I wasn't sure if this mailing list allows adding links to external
> > pictures so here's an attempt at a drawing in plain text (hopefully
> > it doesn't get all messed up).
> >                        From Proxy 1
> >  From local PUBs      on remote nodes
> >         |                 |      |                
> >      bind()         connect() connect()      
> >      ipc://            tcp://  tcp://         
> >         |                 |      |                
> >   +-----+------+       +--+------+--+
> >   |    XSUB    |       |    XSUB    |
> >   +------------+       +------------+
> >   |  Proxy 1   |       |  Proxy 2   |
> >   +------------+       +------------+
> >   |    XPUB    |       |    XPUB    |
> >   +-----+------+       +-----+------+
> >         |                    |
> >       bind()               bind()
> >       tcp://               ipc://
> >         |                    |
> >   To Local SUBs        To Local SUBs
> >  and Proxy 2 on    
> >   remote nodes 
> > The proxies use zmq_proxy_steerable(). I've already tested this and
> > it works. My question is regarding the XSUB socket on "Proxy 2".
> > That's the socket that connects to all the remote nodes. And since
> > I
> > want to be able to add or remove nodes, how can I dynamically
> > change
> > the connections on that socket? I thought of 3 ways I could
> > potentially do this.
> > Run zmq_proxy_steerable() in one thread (Thread1) and run a second
> > thread (Thread2) where I could invoke zmq_connect/disconnect to
> > change the connections. I suspect that it is probably not safe for
> > Thread2 to modify the connections while zmq_proxy_steerable() is
> > running in Thread1. 
> > Thread2 could send a "PAUSE" to Thread1 to make it temporarily stop
> > processing messages. Then Thread2 invokes zmq_connect/disconnect to
> > change the connections. Finally, Thread2 sends "RESUME" to Thread1.
> > Again, not sure this is safe. Can the connections on a socket be
> > changed while zmq_proxy_steerable() is PAUSED?
> > Thread2 could write the list of nodes (IP addresses) to a data
> > struct
> > shared with Thread1. Then Thread2 sends a TERMINATE to Thread1.
> > Thread1 terminates gracefully, reads the new list of nodes, changes
> > the connections on the socket accordingly, then goes back to
> > running
> > zmq_proxy_steerable(). With this I'm not sure what would happen to
> > in-flight messages. That is, when TERMINATE is received by Thread1,
> > does it drop any of the messages it's currently processing?
> > What's the best approach? Is there another (safer) way to do this?
> > 
> > Thanks,
> > Martin
> 
> PUB/SUB Sockets are not thread safe, so you'll need to terminate, do
> the changes you need, and restart the proxy.

And for your last question, PUB/SUB are lossy (by design) so messages
might be lost if the high water-mark (can be configured) is exceeded in
the SUB sockets while the proxy is down.

You can set the HWM to infinite but that means no bounds to memory
utilisation, so you need to be sure that the application can cope (eg:
once the proxy restarts, it's fast enough to recover).

-- 
Kind regards,
Luca Boccassi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20181018/d5c9c8b9/attachment.sig>


More information about the zeromq-dev mailing list