[zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!

Martin Lucina mato at kotelna.sk
Fri Mar 25 20:21:04 CET 2011


dcreager at dcreager.net said:
> OSX uses “two-level namespaces”, which breaks this.  When the linker resolves a symbol, it records which library it found the symbol in, and will always use that library at runtime.  So this example will make the allocations in app.c use tcmalloc, but *not* the allocations in zmq.  You have to also link with tcmalloc when you compile zmq itself.
> 
> You can turn off two-level namespaces at runtime, but then you have the error I mentioned earlier, where the runtime linker itself allocates some memory using the standard malloc, before switching all of the symbols over to tcmalloc.

I read somewhere that tcmalloc has support at least on Windows to detect
free()ing of blocks that were allocated using the system malloc and passing
those to the system free. Maybe you can investigate such an approach on OSX
in the interim.

> In this case, it's definitely an annoying feature to have to work around — but it's not really a “bug” per se, that we can ask Apple to fix — it's how they've chosen to design their linker.
> 
> > Martin, maybe all we need to do is document this prominently in the FAQ? It
> > is a *much less invasive and complex* solution than adding custom
> > allocators to 0mq!
> 
> Less invasive to whom?  For the people working on zmq internals, you just have to train yourself to write zmq_malloc instead of malloc.  For the people using zmq, there are two extra parameters to zmq_init, which you can leave as NULL if you want the default system allocator.
> 
> I guess it's a matter of taste.  There's a specific behavior I need to achieve: use a different allocator than the one provided by system.  Personally, I'd rather be able to express this behavior directly in the code itself, rather than having to play games with the linker to make it happen.

Ok, so there is a need for a custom allocator on the critical path rather
than just replacing malloc/free on a library/application-wide basis.

Ultimately that comes down to two things:

1) Designing the custom allocator API in such a way that it is not invasive
to the majority of users that don't have a need for it. This means, as
Pieter says elsewhere, that users not needing such functionality should not
have to e.g. add an extra parameter to all zmq_msg calls.

2) Someone has to do the work. Again, as has already been mentioned, this
is not as trivial as defining zmq_malloc() since libzmq is implemented in
C++.

-mato



More information about the zeromq-dev mailing list