[zeromq-dev] rc == 0 (./zmq/mutex.hpp:94)

[Martin Sustrik]

Hi Aamir,

> I applied the patch and it fixed the bug! Thanks!
> 
> I also tested this problem in x86 by including C++ compiler flag
> -DZMQ_FORCE_MUTEXES ... as expected, the error occurs on x86 as well.
> And it is resolved by applying your patch.

Great. We'll refine the patch (I would like to get rid of the extra 
memory allocation involved) and merge it into the trunk.

> Attached is a cleaned-up version of the example code. Is there any
> practical difference between putting msg_out inside the while loop
> versus outside (other than possible performance improvement by putting
> it outside)? Is there any pitfall to be aware of if msg_out is
> declared outside the while loop in broker_loop and then re-used?

As far as I can say the only difference is the performance impact of 
message object being cleaned twice (once in send function, once in 
destructor) rather than once (in send function). However, I would except 
the impact to be negligible as it involves only a simple memory write on 
a location that has almost 100% chance of being in CPU cache.

> On a related note ....
> 
> Do you know off-hand what kind of performance gains are possible by
> using the inline atomic assembly instructions instead of pthread
> mutexes?

It heavily depends on scenario. When messages happen to be sent in 
batches the performance impact is diluted (only one sync operation per 
batch). If the timing of messages is such that they are sent one-by-one, 
the impact will be more severe. Also, the more CPU cores/threads are 
involved the better atomic operations scale when compared to mutexes.

> Let us say that the performance gain is significant and let us further
> say that I can find the resources to get a PowerPC lock-free version
> of atomic_bitmap.hpp, atomic_counter.hpp and atomic_ptr.hpp ... Would
> you guys have any way of incorporating the ppc inline assembly into
> the 0MQ trunk? There could be testing problems if none of the 0MQ
> developers has a PowerPC chip to work on? .. You could potentially use
> a PlayStation3 with Linux to do the testing ... PS3 is a relatively
> cheap PowerPC environment, though setting up the build environment
> might be a pain.

That would be great. We have no experience with PPC so it would be 
rather hard for us to code the atomic operations ourselves. However, for 
a skilled person it shouldn't take more than few hours of work. In any 
case, if you decide to go this way, let's discuss it beforehand as there 
are non-trivial assumptions about CPU's memory model.

Martin