[zeromq-dev] Parallel sort
oliver at kfs.org
Tue Aug 3 08:13:43 CEST 2010
On 8/2/2010 7:32 PM, Pieter Hintjens wrote:
> Uhm, that'd be 1+n sockets per thread where n might be 2.
With mechanisms like these, you need to be careful where you choose
between scalability (off-device distribution) and performance.
See this post about one of Facebook's developers experimenting with
Parallel sort in Erlang.
Using lock-based parallelism, I expect a quite drastic performance
improvement by using a parallel sort over a serial sort. GCC 4.4+ comes
with STL's parallel/algorithm which includes a parallel_sort.
I think his primary problem was the ratio of compute time to management
time and setting too low a grain size for "go ahead and do this locally".
More information about the zeromq-dev