[zeromq-dev] Thread Safe sockets

[john skaller]

On 07/02/2012, at 11:05 AM, Nadav Samet wrote:

> Hi John, 
> 
> Your argument is valid, but it can be equally applied to std::queue, std::map or any other stateful object.

Agree. In principle, you're right.

> As a single data point, the C++ STL standard chose that the clients will provide the locking. There are various reasons in http://www.sgi.com/tech/stl/thread_safety.html, although I believe their performance argument does not apply in our case.

The real reason is very simple: C++90 Standard didn't support threads at all in its object model,
so there was no way to even consider a thread-safe data structure. None of the other reasons
matter :)

> I tend to agree with them that wrapping classes in a thread-safe manner on the client is easy and can provide more encapsulation.

Actually that is not the case because of the categorical problem mentioned previously.
Haskell could probably do this right, possibly with Monads. The problem is:

Suppose we have Vector and we make a TS_Vector which is thread safe.

Now suppose we have

	typedef VVector<T> = Vector < Vector <T> >

The problem is the TS wrapper doesn't commute:

	Vector <TS_Vector<T> > != TS_Vector <Vector T> > != TS_Vector <TS_Vector<T> >

 What we actually *require* here is a natural transformation that does commute.

If you're having trouble grokking this, there's another example of it in C++, 
namely "const". This is a serious design error in C++, const is a very bad idea
because it doesn't commute. In other words, const array is not the same as
array of const.

You can see this problem in STL where there are separate "const_iterator".
It's all badly broken.

> 
> If libzmq is changed such that sending/receiving individual frames is no longer possible for clients, then I think it's reasonable to have locking inside zeromq. But as long as users talk to the API in terms of single frames, the current implementation does not really provide thread safety and might cause confusing problems to users.

The API as I modified it does not provide safety over multi-part messages.
But that's nothing new, it didn't before either. It does provide safety for each
frame/fragment/message part (or whatever you want to call those bits).

There is one thing that might be interesting. Hmm.. There is a lock now *inside*
the socket object where is needs to be.

So you could use non-safe sockets and the API:

	lock(socket)
	unlock(socket)

This would work for multi-part messages. The functions would have to be called by the
user, and only on un-safe sockets. In fact, an alternative is to provide these functions
and get rid of the automatic locking based on the context of a flag in the socket.

ALTERNATIVE TS proposal:

	void *lock(void *socket_t) // returns locked socket, arg must not be locked
	void *unlock(void *socket_t) // returns unlocked socket, arg must be locked

This puts locking back in the hands of the user BUT it keeps the mutex inside
the object so the user doesn't have to worry about keeping the socket/mutex
pairs associated.

--
john skaller
skaller at users.sourceforge.net