[zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!
sustrik at 250bpm.com
Fri Mar 25 07:14:34 CET 2011
As already noted there are 2 disctinct problems. One is using a better
performing allocator for everything to alleviate performance issues in
built-in allocator provided by OS, the other is using a specific
user-supplied allocator for message bodies.
Let's focus on the former now, ignoring the latter for a while so that
the discussion doesn't get confused.
1. The performance problem (along the CPU/memory tradeoff) happens on
process level (process consuming too much memory etc.) not on library
level. All the libraries loaded to the process contribute to the
problem. Thus, the solution should be preferably at process level.
2. One option would be tweaking the OS settings in such a way as to
optimise built-in allocator behaviour. Your OS may or may not provide
3. Another option is to intercept the allocation calls is OS loader by
providing alternate implementations to the allocation-related functions.
That's the way jemalloc and tcmalloc do it.
4. It turns out that the latter doesn't work well on OSX due to the OS
5. The question to answer is: Do we want to patch 0MQ to compensate for
buggy OSX? And do we at least understand the bugs so that we are sure
they can't be worked around?
6. If so, the only way to replace standard allocator is at the compile
time. Aside of OS loader that's the only phase that happens before any
allocation is done.
7. We should then find out whether it's possible to overload standard
malloc/free in the library in such a way that our implementation is used
instead of the implementation provided by C runtime.
8. If that's not possible, we'll have to replace all allocations in the
codebase by custom ones. This includes:
a. Replacing all malloc/free calls.
b. Replacing all new/delete calls by hand-written new-like [template]
c. Pass custom allocator to all STL structures in the codebase.
More information about the zeromq-dev