[zeromq-dev] can I send messages to connect socket from multiple threads with mutex protection?

anand n anand.pathivu at gmail.com
Thu Jan 3 11:58:22 CET 2019


Thanks Juergen for sharing your inputs. Understood your point and totally
agree with you. I'm evaluating few other options to meet my requirement.

Regards,
Anand.

On Thu, Jan 3, 2019 at 3:25 PM Juergen Gnoss <jgnoss at hotmail.com> wrote:

>
> The question is not
> "Do you expect any problem ... the application shares a zmq-pair connect
> socket with multiple threads"
>
> The question is, Is it right to do that.
> There the answer is clearly no.
>
> You should do an application design by looking at the applications future.
>
> I personally don't want to keep always the weak points of my (bad
> designed)
> application in mind while I'm extending it.
>
> ju
>
> ------------------------------
> *From:* zeromq-dev <zeromq-dev-bounces at lists.zeromq.org> on behalf of
> anand n <anand.pathivu at gmail.com>
> *Sent:* Wednesday, January 2, 2019 10:25 PM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] can I send messages to connect socket from
> multiple threads with mutex protection?
>
> Thanks a lot Bill, Juergen and Luca for your quick response and
> suggestions. I am looking at them.
>
> I have few more questions/clarifications on the zmq-pair socket type
> (please see below) and want to say thanks in advance for your kind reply!
>
> 1. I see that the word 'experimental' is dropped in the latest API
> documentation (http://api.zeromq.org/4-2:zmq-socket) for ZMQ_PAIR socket
> type. It just states that functionality such as 'auto-reconnection' is not
> implemented. So shall I safely assume that the socket type is official now
> and the basic functionality is expected to work fine without any issues?
>
> 2. I understand that it is not recommended to use non-thread safe legacy
> zmq sockets in more than one thread. However just for my understanding, I
> am asking this question. Do you expect any problem from zmq implementation
> perspective, if the application shares a zmq-pair connect socket with
> multiple threads only to send message to a single peer thread (with mutex
> around zmq_send()) and doesn't use the socket in zmq_poll(). The socket
> will never be disconnected/destroyed by the application process once
> created and lives forever until the process exits.
>
> I took a quick look at thread safe client-server socket type and it looks
> that it will address my requirement. I will explore more on that and come
> back if I have any question.
>
> Thanks,
> Anand.
>
> On Mon, Dec 31, 2018 at 9:42 PM Juergen Gnoss <jgnoss at hotmail.com> wrote:
>
> Hi Anand
>
> Sounds you've already spent a fair amount of time in your project, but
> anyway
> here some of my experiences.
>
> Time ago I was on the same point as you, and I gave the DRAFT stuff of CZMQ
> a shot. Really exciting stuff that, and since then I kept using that.
>
> Putting all that API thread based stuff in zactors, keeping the inproc
> communication
> on the internal pipe and put external communication on additional
> client/server sockets
> in the actors gives a really clean design.
>
> And yes, it's thread safe that way.
>
> ju
>
> ------------------------------
> *From:* zeromq-dev <zeromq-dev-bounces at lists.zeromq.org> on behalf of
> Bill Torpey <wallstprog at gmail.com>
> *Sent:* Monday, December 31, 2018 10:55 AM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] [C/Linux/ZMQ_PAIR/Unidirectional-flow] - can
> I send messages to connect socket from multiple threads with mutex
> protection?
>
> Hi Anand:
>
> Please see comments below.  Good luck with your project!
>
> Bill
>
> > On Dec 30, 2018, at 10:45 PM, anand n <anand.pathivu at gmail.com> wrote:
> >
> > Team,
> >
> > I am building a multi threaded application in linux with C as the
> programming language and use ZMQ_PAIR sockets for inter-thread
> communication (zmq version - 4.3.0). The main thread of the application
> communicates with external applications using router-dealer model.
> >
> > The application process spawns multiple child threads for handling
> specific functionalities. Each thread creates two ZMQ_PAIR sockets (one for
> bind & the other for connect) and bind & connect these sockets to same end
> point (inproc://<unique-name-per-thread>). All these are done during
> process initialization and finally the threads invoke zmq_poll on the bind
> socket. The connect socket is only used for sending messages to the thread
> and the bind socket is used to receive them.
>
> In my experience, PAIR sockets can be unreliable (
> https://github.com/zeromq/libzmq/issues/2759).  I published test code
> here (https://github.com/WallStProg/zmqtests/tree/master/threads) that
> demonstrates the problems.  What I ended up doing was using CLIENT/SERVER
> sockets instead of PAIR sockets, which have proved reliable in my testing.
>
> Unlike PAIR sockets, SERVER sockets are not exclusive, so a single SERVER
> socket can receive messages from any number of client sockets (i.e., it is
> one-to-many rather than one-to-one as with PAIR sockets).
>
> Another benefit of CLIENT/SERVER sockets is that they are thread-safe,
> which leads to my next comment…
>
> >
> > The thread publishes APIs to other threads and messages are sent to the
> connect socket of the thread from these APIs (in non-blocking mode). Of
> course socket access (i.e, zmq_send) is mutex protected from application
> perspective.
>
> In general, protecting “legacy” ZMQ sockets with mutexes is not
> recommended. While it may be theoretically possible to make synchronization
> work with mutexes, in practice that can be tricky, particularly if the
> application uses zmq_poll to wait on socket events.
>
> Better to restrict socket access to a single thread, or to use one of the
> newer thread-safe socket types if possible.
>
> >
> > I've tested the application and its working fine so far without any
> issues. I understand that zmq socket is not thread safe and as you see, in
> this model, messages are sent to the connect socket from multiple threads
> (not concurrently anyway due to the mutex protection!).
>
> I think you may be confusing endpoints and sockets — with some socket
> types (CLIENT/SERVER, PUB/SUB), many sockets can connect to a single
> endpoint, and in those cases any necessary synchronization is handled
> internally by ZMQ.
>
>
> > Do you think that sharing the socket in this way can cause any problem
> from zmq perspective?
> >
> > Please let me know if something is not clear or if you need more info.
> Thanks a lot for your help.
> >
> > Regards,
> > Anand.
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20190103/34ab834f/attachment.htm>


More information about the zeromq-dev mailing list