[zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!

Chuck Remes cremes.devlist at mac.com
Sat Mar 26 00:39:39 CET 2011


On Mar 25, 2011, at 4:29 PM, Jason Evans wrote:

> Chuck Remes <cremes.devlist <at> mac.com> writes:
>> I ran a *long* test (2 days) using the jemalloc allocator on linux (i.e.
> replacing the allocator at the
>> process level).
>> 
>> It improved the usage and allocation of heap memory by 0mq by about 50%. The
> heap still slowly grew due to
>> fragmentation, but with jemalloc the growth was about twice as slow as the
> standard glibc malloc. This is a
>> partial win.
>> 
>> Unfortunately, jemalloc didn't solve the problem.
>> 
>> Looking deeper into the issue, it looks like the long-term fix is to use a
> SLAB allocator [1]. [...]
>> 
>> [1] http://en.wikipedia.org/wiki/Slab_allocation
> 
> jemalloc *is* a slab allocator, so you're unlikely to improve the situation much
> by layering a sub-allocator on top of it.  More likely, this change would
> actually increase fragmentation, for the same basic reason that using multiple
> arenas in jemalloc increases fragmentation (available memory within a slab from
> arena 0 is not available to a thread that is assigned to arena 1).

If anyone would know, it would be you. :)

Since that's the case, then I really don't know what to try next to eliminate this slow "leak" that I have in my apps that use 0mq. Until I or someone else has a new bright idea, I'll just plan to restart the system every 3 days to release back all of the memory they have grabbed.

cr




More information about the zeromq-dev mailing list