[zeromq-dev] Thread safety of the ROUTER socket

[Bruno D. Rodrigues]

I’m gonna answer from my own experience but I’m sure someone with better knowledge will correct me ;)

1. Albeit I’m using java and even the jeromq codebase, I’ve solved it by simply synchronizing the access to the socket.recv and socket.send (mutex, whatever it’s called on your language), and by ensuring that the whole multipart is read in one go. In other words, I have a wrapper that sends or receives a list of parts, and the code locks the socket and sends/receives all multi-part messages in one go. (for completeness, I have another helper that allows to send and receive multiple multi-part messages, so it could send or receive multiple messages without the need to lock the socket for each (multipart) message.

2/3. from my experience with the zmq_proxy codebase I don’t think it’s possible to have one thread send’ing and a different one recv’ing. I’ve tried this with the proxy and it crashed with assertion errors. The poll() is awfully slow on the Mac, so one solution I tried was spewing two threads, one send(backend, recv(frontend)) and vice-versa, and it died. So i guess involving poll would make it even worse.

One elegant solution is to chain sockets in a way that only one thread is responsible for the main socket, and communicate with the multiple “workers” via two inproc sockets. It’s quite elegant, and, at least in java, way more performant than blockingqueues!
 

On Nov 29, 2013, at 14:51, Asbjørn Støen <asbjorn.stoen at gmail.com> wrote:

> Hi,
> 
> in my Java app, I'm serving long-running requests from multiple clients on a ROUTER socket. To achieve concurrency, I have one thread reading the socket, and then I dispatch the request to a pool of worker threads. This means that replies will be produced some time in the future by other threads than the reader thread.
> 
> I'm aware that ØMQ sockets are not thread-safe, and of course, two threads cannot send replies simultaneously to the same socket.
> 
> However, I would like some clarification on the concurrency of recv() and send() operations:
> 
> 1. Do I need to perform the recv() and send() sequentially (or preferrably in one thread), as the doc seems to have it? Or:
> 2. Can I safely poll the socket for incoming messages while concurrently sending a message from another thread? Or:
> 3. Can I safely receive messages in one thread while concurrently sending a message from another thread?
> 
> The problem with #1 above is that polling the socket would block replies from being sent for the duration of the poll. Non-blocking polls are of course subject to spinlock. I'm hoping that at least #2 is true... although #3 should be expected from an asynchronous socket.
> 
> Regards,
> Asbjørn
> 
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev