[zeromq-dev] spin lock versus mutex

[Martin Sustrik]

Hi Steven,

>   In mutex.hpp spin locks are being for Win32 but not for POSIX, you
> might see a performance increase by changing POSIX to use spin locks
> too, although more likely you should implement both a zmq::mutex_t and
> zmq::spinlock_t and decide on a case by case basis which is the best
> to use.
> 
> Spin locks on Win32:
> EnterCriticalSection(), LeaveCriticalSection()
> 
> Spin locks on Posix:
> pthread_spinlock_lock(), pthread_spinlock_unlock()
> 
> Mutexes on Win32:
> WaitForSingleObject(), ReleaseMutex()
> 
> Mutexes on Posix:
> pthread_mutex_lock(), pthread_mutex_unlock()
> 
> I also noticed when using Conditions Win32 needs a spin lock but posix
> needs a mutex, however I only use them to implement a read-write lock
> on XP whereby Vista+ have a Condition API.
> 
> I saw a performance increase in OpenPGM making the change, but some
> locks also completely froze, I don't know if that means a pthread
> memory barrier needs to be used at the same time as the critical
> section is very small.

Mutexes are currently used in 2 different scenarios:

1. To wait till some event happens (like message is received). This 
functionality is used on the critical path. Waiting period is unlimited. 
Performance is critical.

2. To synchronise access to shared data. This functionality is never 
used on critical path. Waiting period is very short. Performance is not 
really important.

The first one requires a real mutex/semaphore and is encapsulated in 
zmq::simple_semaphore_t class. On Win32 it uses CreateEvent et al. - 
AFAICS it's not a spinlock, which is correct. On POSIX it's mutex, which 
is OK as well.

The second one in zmq::mutex_t and it could do with a spinlock, however, 
it's out of critical path, so the performance degradation due to mutex 
shouldn't matter. On Win32 it used CriticalSecion i.e. spinlock. On 
POSIX, mutex is used which is suboptimal. If there's anyone out there 
who cares fixing it, give it a try.

Martin