[zeromq-dev] Parallel sort

[Oliver Smith]

On 8/2/2010 7:32 PM, Pieter Hintjens wrote:
> Uhm, that'd be 1+n sockets per thread where n might be 2.
>    

With mechanisms like these, you need to be careful where you choose 
between scalability (off-device distribution) and performance.

See this post about one of Facebook's developers experimenting with 
Parallel sort in Erlang. 
http://yarivsblog.com/articles/2009/03/09/parallel-merge-sort-in-erlang/

Using lock-based parallelism, I expect a quite drastic performance 
improvement by using a parallel sort over a serial sort. GCC 4.4+ comes 
with STL's parallel/algorithm which includes a parallel_sort.

I think his primary problem was the ratio of compute time to management 
time and setting too low a grain size for "go ahead and do this locally".