[zeromq-dev] C++ assertion failed with Java client

[Chuck Remes]

On Feb 2, 2012, at 6:03 AM, john skaller wrote:

> I would put these directly in the official ZMQ C API.
> 
> The rationale is that ZMQ's own rationale is that you can program with ZMQ
> instead of using all those nasty locks, share memory barriers, and all the
> other things that make multi-threaded programming a nightmare.
> 
> In Felix I have a garbage collector: flx_collector_t and a second collector
> flx_ts_collector_t which is a thread-safe wrapper around the first one: both are
> derived from flx_gc_t abstract class in the C++ code. 
> 
> The wrapping should be done in C++ and exported to C, not in C around
> the C API. IMHO. Which means it has to be part of the ZMQ C API standard.
> 
> The reason is I think
> 
> (a) the locking can be made somewhat "conditional" and
> 
> (b) because of the particular structure of ZMQ other techniques
> which can avoid mutex or semaphores may be used.
> 
> (c) it makes the Windows and Posix usage of ZMQ the same
> in an environment where 0MQ sockets are used in several threads,
> rather than forcing the client to use OS local locking techniques.
> 
> 
> It's not clear if the "right way" is to have a zmq_context, and a 
> zmq_ts_context, so that if you make the latter all the socket
> objects spawned off it are thread safe. This is clearly simpler
> and avoids changes to the other functions, but it provides
> less control and therefore imposes an overhead.

I think this suggestion is very misguided. I also infer from your suggestion that you haven't really grok'ed the essence of 0mq.

There is a simple Rule for using 0mq with threads. That Rule is:

*Only* use a socket from the thread that created it.

If you follow that Rule, 0mq and threads get along swimmingly. There is no need for complicating the API or making the internals to the library more complex.

I use 0mq in a highly multi-threaded application. I follow The Rule and have had *no problems* with scaling. When I need to communicate between threads, I use 0mq sockets to handle the message passing for me (via inproc transport). I have successfully used this technique to coordinate work between 48+ live threads and over 30k active sockets across a LAN in my cluster. I don't want libzmq made more complex and *slow* just because it takes a little extra effort and design to follow The Rule.

For those *few* occasions where it is imperative that multiple threads share a single socket, a mutex may be used to protect it. This is *standard practice* for sharing any data structure amongst threads so it shouldn't be a surprise. I have used this technique in some small utility programs and it works well enough without any additional in-library support.

Lastly, I think it's time to trot out an old blog post that Martin Sustrik wrote on this topic.

http://www.zeromq.org/blog:multithreading-magic

cr