[zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!

[Martin Sustrik]

Hi all,

As already noted there are 2 disctinct problems. One is using a better 
performing allocator for everything to alleviate performance issues in 
built-in allocator provided by OS, the other is using a specific 
user-supplied allocator for message bodies.

Let's focus on the former now, ignoring the latter for a while so that 
the discussion doesn't get confused.

1. The performance problem (along the CPU/memory tradeoff) happens on 
process level (process consuming too much memory etc.) not on library 
level. All the libraries loaded to the process contribute to the 
problem. Thus, the solution should be preferably at process level.

2. One option would be tweaking the OS settings in such a way as to 
optimise built-in allocator behaviour. Your OS may or may not provide 
such option.

3. Another option is to intercept the allocation calls is OS loader by 
providing alternate implementations to the allocation-related functions. 
That's the way jemalloc and tcmalloc do it.

4. It turns out that the latter doesn't work well on OSX due to the OS 
loader brittleness.

5. The question to answer is: Do we want to patch 0MQ to compensate for 
buggy OSX? And do we at least understand the bugs so that we are sure 
they can't be worked around?

6. If so, the only way to replace standard allocator is at the compile 
time. Aside of OS loader that's the only phase that happens before any 
allocation is done.

7. We should then find out whether it's possible to overload standard 
malloc/free in the library in such a way that our implementation is used 
instead of the implementation provided by C runtime.

8. If that's not possible, we'll have to replace all allocations in the 
codebase by custom ones. This includes:

a. Replacing all malloc/free calls.
b. Replacing all new/delete calls by hand-written new-like [template] 
functions.
c. Pass custom allocator to all STL structures in the codebase.

Martin