[zeromq-dev] Important: backward incompatible changes for 0MQ/3.0!

[Robert G. Jakabosky]

On Thursday 24, Douglas Creager wrote:
> >> 1. If tcmalloc et al are brittle, do we understand why is it so? Is it
> >> an inherent problem or just a lame implementation?
> 
> It wasn't itself tcmalloc that was brittle, it was Mac's equivalent of
> LD_PRELOAD.
> 
> >> 2. Douglas' patch stores the pointer to allocator function in a global
> >> variable. That breaks the model of separate contexts (i.e. that several
> >> instances of 0mq can exist in a single process without misinteracting
> >> with each other). We have to think of something more resilient.
> > 
> > Perhaps the allocator should be embedded into the context. You could even
> > create separate contexts if you wish to have different allocators.
> 
> Exactly.  That eliminates the global variable nicely.  But you've have to
> add a context parameter to the zmq_msg_t functions.
> 
> >> 3. There are at least two use cases here. AFAIU, Douglas want to
> >> custom-allocate the message bodies while Chunk want to custom-allocate
> >> everything. We should think a bit more about whether one use case
> >> subsumes the other or whether they are two separate use cases with two
> >> separate solutions.
> > 
> > Maybe you could provide two args to the allocator when allocating memory:
> > the size in bytes and a hint. This hint could be used by the allocator
> > to decide upon it: use different memory pools for different parts of the
> > program, etc. Then, you could use the allocator for everything in ZeroMQ
> > itself and in programs using ZeroMQ, assuming ZeroMQ will have a default
> > implementation of the allocator.
> > 
> > If we go this route, it could also be the chance to use a couple of
> > macros for interfacing with the allocator:
> > 
> > ZMQ_ALLOC(size, hint);
> > ZMQ_REALLOC(old_size, new_size, hint);
> > ZMQ_FREE(size, hint);
> > 
> > The point is that the macros can (maybe only in debugging mode) decorate
> > the call by passing the file name and line number to the allocator. I
> > have found this invaluable when debugging memory problems.
> > 
> > Unluckily, it has been too long since the last time I did something like
> > this; I would have to force myself to get back up to speed in order to
> > contribute with more than simple ideas.
> 
> The basic API in my previous patch would be a good starting point for this
> — it defined those macros, like you suggest.  You provide a custom
> allocator as a single function:
> 
>   void * (*alloc_func)(void *user_data, void *old_ptr, size_t old_size,
> size_t new_size);
> 
> Depending on which parameters you pass in, this mimics malloc, realloc, and
> free.  This API is the same one they use in the Lua interpreter, and I
> implemented the same thing in the Avro C bindings, too.  It seems to work
> pretty well.

+1,  I have also used the same allocator interface for one of my private 
projects.

> 
> The main issue with this API is that you have to keep track of the
> allocated size of everything you allocate.  For most fixed-size types,
> that's not an issue, since you can just use sizeof when you free, just
> like when you malloc.  For variable-sized buffers, though, you usually
> need an extra field to store the allocated size.

For variable-sized buffers/array you normally need to keep the size anyways.

> Another issue is that the zmq_msg_t API is all pure C, whereas most of the
> rest of the library is C++, so we'll have to overload the definitions of
> new and delete to really make this pervasive.
> 
> I'm not sure about the hint parameter you mention — can you explain what
> you mean by that?  I had a user_data parameter in the allocation function,
> but this was passed in by the user of the library.  Your macros look like
> the hint would come from within 0mq...so would we have some well-defined
> hints that tell the allocator which part of the library the memory is for?

A hint parameter could be used to tell the allocator that the memory block 
might need to grow later, or is fixed size and will never grow.  For fixed 
sized block the allocator can allocate the memory from a packed region, for 
grow-able memory blocks the allocator can try to leave some space after the 
block for fast resizing.  You may also want to mark some memory as temporary 
blocks that will be freed very soon.

The hint parameter could also identify allocations for message blocks vs. 
other allocations from 0MQ.

-- 
Robert G. Jakabosky