[zeromq-dev] can I send messages to connect socket from multiple threads with mutex protection?

anand n anand.pathivu at gmail.com
Thu Jan 3 04:25:24 CET 2019

Thanks a lot Bill, Juergen and Luca for your quick response and
suggestions. I am looking at them.

I have few more questions/clarifications on the zmq-pair socket type
(please see below) and want to say thanks in advance for your kind reply!

1. I see that the word 'experimental' is dropped in the latest API
documentation (http://api.zeromq.org/4-2:zmq-socket) for ZMQ_PAIR socket
type. It just states that functionality such as 'auto-reconnection' is not
implemented. So shall I safely assume that the socket type is official now
and the basic functionality is expected to work fine without any issues?

2. I understand that it is not recommended to use non-thread safe legacy
zmq sockets in more than one thread. However just for my understanding, I
am asking this question. Do you expect any problem from zmq implementation
perspective, if the application shares a zmq-pair connect socket with
multiple threads only to send message to a single peer thread (with mutex
around zmq_send()) and doesn't use the socket in zmq_poll(). The socket
will never be disconnected/destroyed by the application process once
created and lives forever until the process exits.

I took a quick look at thread safe client-server socket type and it looks
that it will address my requirement. I will explore more on that and come
back if I have any question.


On Mon, Dec 31, 2018 at 9:42 PM Juergen Gnoss <jgnoss at hotmail.com> wrote:

> Hi Anand
> Sounds you've already spent a fair amount of time in your project, but
> anyway
> here some of my experiences.
> Time ago I was on the same point as you, and I gave the DRAFT stuff of CZMQ
> a shot. Really exciting stuff that, and since then I kept using that.
> Putting all that API thread based stuff in zactors, keeping the inproc
> communication
> on the internal pipe and put external communication on additional
> client/server sockets
> in the actors gives a really clean design.
> And yes, it's thread safe that way.
> ju
> ------------------------------
> *From:* zeromq-dev <zeromq-dev-bounces at lists.zeromq.org> on behalf of
> Bill Torpey <wallstprog at gmail.com>
> *Sent:* Monday, December 31, 2018 10:55 AM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] [C/Linux/ZMQ_PAIR/Unidirectional-flow] - can
> I send messages to connect socket from multiple threads with mutex
> protection?
> Hi Anand:
> Please see comments below.  Good luck with your project!
> Bill
> > On Dec 30, 2018, at 10:45 PM, anand n <anand.pathivu at gmail.com> wrote:
> >
> > Team,
> >
> > I am building a multi threaded application in linux with C as the
> programming language and use ZMQ_PAIR sockets for inter-thread
> communication (zmq version - 4.3.0). The main thread of the application
> communicates with external applications using router-dealer model.
> >
> > The application process spawns multiple child threads for handling
> specific functionalities. Each thread creates two ZMQ_PAIR sockets (one for
> bind & the other for connect) and bind & connect these sockets to same end
> point (inproc://<unique-name-per-thread>). All these are done during
> process initialization and finally the threads invoke zmq_poll on the bind
> socket. The connect socket is only used for sending messages to the thread
> and the bind socket is used to receive them.
> In my experience, PAIR sockets can be unreliable (
> https://github.com/zeromq/libzmq/issues/2759).  I published test code
> here (https://github.com/WallStProg/zmqtests/tree/master/threads) that
> demonstrates the problems.  What I ended up doing was using CLIENT/SERVER
> sockets instead of PAIR sockets, which have proved reliable in my testing.
> Unlike PAIR sockets, SERVER sockets are not exclusive, so a single SERVER
> socket can receive messages from any number of client sockets (i.e., it is
> one-to-many rather than one-to-one as with PAIR sockets).
> Another benefit of CLIENT/SERVER sockets is that they are thread-safe,
> which leads to my next comment…
> >
> > The thread publishes APIs to other threads and messages are sent to the
> connect socket of the thread from these APIs (in non-blocking mode). Of
> course socket access (i.e, zmq_send) is mutex protected from application
> perspective.
> In general, protecting “legacy” ZMQ sockets with mutexes is not
> recommended. While it may be theoretically possible to make synchronization
> work with mutexes, in practice that can be tricky, particularly if the
> application uses zmq_poll to wait on socket events.
> Better to restrict socket access to a single thread, or to use one of the
> newer thread-safe socket types if possible.
> >
> > I've tested the application and its working fine so far without any
> issues. I understand that zmq socket is not thread safe and as you see, in
> this model, messages are sent to the connect socket from multiple threads
> (not concurrently anyway due to the mutex protection!).
> I think you may be confusing endpoints and sockets — with some socket
> types (CLIENT/SERVER, PUB/SUB), many sockets can connect to a single
> endpoint, and in those cases any necessary synchronization is handled
> internally by ZMQ.
> > Do you think that sharing the socket in this way can cause any problem
> from zmq perspective?
> >
> > Please let me know if something is not clear or if you need more info.
> Thanks a lot for your help.
> >
> > Regards,
> > Anand.
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20190103/c79cc1ab/attachment.htm>

More information about the zeromq-dev mailing list