[zeromq-dev] Can zmq_connect/zmq_disconnect be invoked on a socket while zmq_proxy_steerable() is running in a separate thread for the same socket?

Martin.Belanger at dell.com Martin.Belanger at dell.com
Thu Oct 18 17:05:00 CEST 2018

​I'm designing a multi-node pub-sub pattern (each node is a separate computer running Debian Linux). The number of nodes is configurable and can change over time (i.e. through configuration I can add or remove nodes). I want to use XPUB-XSUB proxies on each node as depicted below. The following link is for a picture showing the whole topology: https://docs.google.com/drawings/d/1WZ-h_3cP1wc0WEEBmtrhH6CDNakyNE4KTb-GApIV7Pk/edit?usp=sharing

I wasn't sure if this mailing list allows adding links to external pictures so here's an attempt at a drawing in plain text (hopefully it doesn't get all messed up).
                       From Proxy 1
 From local PUBs      on remote nodes
        |                 |      |                
     bind()         connect() connect()      
     ipc://            tcp://  tcp://         
        |                 |      |                
  +-----+------+       +--+------+--+
  |    XSUB    |       |    XSUB    |
  +------------+       +------------+
  |  Proxy 1   |       |  Proxy 2   |
  +------------+       +------------+
  |    XPUB    |       |    XPUB    |
  +-----+------+       +-----+------+
        |                    |
      bind()               bind()
      tcp://               ipc://​
        |                    |
  To Local SUBs        To Local SUBs
 and Proxy 2 on    
  remote nodes ​
The proxies use zmq_proxy_steerable(). ​I've already tested this and it works. My question is regarding the XSUB socket on "Proxy 2". That's the socket that connects to all the remote nodes. And since I want to be able to add or remove nodes, how can I dynamically change the connections on that socket? I thought of 3 ways I could potentially do this.
Run zmq_proxy_steerable() in one thread (Thread1) and run a second thread (Thread2) where I could invoke zmq_connect/disconnect to change the connections. I suspect that it is probably not safe for Thread2 to modify the connections while zmq_proxy_steerable() is running in Thread1. 
Thread2 could send a "PAUSE" to Thread1 to make it temporarily stop processing messages. Then Thread2 invokes zmq_connect/disconnect to change the connections. Finally, Thread2 sends "RESUME" to Thread1. Again, not sure this is safe. Can the connections on a socket be changed while zmq_proxy_steerable() is PAUSED?
Thread2 could write the list of nodes (IP addresses) to a data struct shared with Thread1. Then Thread2 sends a TERMINATE to Thread1. Thread1 terminates gracefully, reads the new list of nodes, changes the connections on the socket accordingly, then goes back to running zmq_proxy_steerable(). With this I'm not sure what would happen to in-flight messages. That is, when TERMINATE is received by Thread1, does it drop any of the messages it's currently processing?
What's the best approach? Is there another (safer) way to do this?


More information about the zeromq-dev mailing list