[zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!

[Chuck Remes]

On Mar 24, 2011, at 8:20 AM, Douglas Creager wrote:

>> The memory allocator should be extensible (think of an allocator strategy); same thing for the disk swap. What I mean by "strategy" is that it should be possible to plug one's own implementation of these things (memory allocator, disk swapper). Both should have default internal implementations.
> 
> Awhile back I submitted a patch to allow a custom memory allocator for zmq_msg_t objects.  It wasn't pervasive throughout the entire library, though.  To extend it to the rest of 0mq, you could store pointers to the allocator and swapper implementations inside of the context object.  But you'd need to change the zmq_msg_t API to take in a context object, since those functions would need access to the allocator.
> 
> The patch wasn't merged into mainline, partly because several people pointed out that you can use a malloc replacement library (tcmalloc, jemalloc, etc) instead.  Then you still call malloc/free/etc in the 0mq code.  I experimented with this a bit, and it actually seemed pretty brittle, at least on Mac OS X — it seems like something allocates memory using the system malloc before loading in the malloc replacement, which causes problems when that memory is freed at the end.  I didn't really follow up on this (other things were more urgent), but my hunch is that an actual allocation/swapper API would be less troublesome.

+1 for this idea.

By adding this feature, I think it eliminates the "wish list" item I added to the 3.0 roadmap under "Internal Memory Allocator." Instead of writing a small block caching & reuse algo into the 0mq library itself, we can allow programmers to customize the memory allocation with their own hand-rolled code or by injecting another malloc library like tcmalloc or jemalloc.

cr