[zeromq-dev] On hooking memory allocations

[Auer, Jens]

Hi,

I don't see a big problem with the C API except that C doesn't support overloads. So if the function has a new name, e.g. zmq_ctx_new_with_allocator, everything stays plain C. The default instance would be a 

static void* malloc_(size_t n, void*) {return malloc(n);}
static void free_(void* ptr, size_t n, void*) {free(ptr);}

allocator_t alloc{
NULL,
malloc_,
free_
};

context_t then stores the member and gets methods to forward memory allocations to the function pointers, passing the hint pointer as an additional argument.

In my C++ code, I can then use an allocator
static void* allocate(size_t n, void* obj) {return static_cast<std::allocator<char>>(obj)->allocate(n); }
static void free_(void* ptr, size_t n, void*obj) { static_cast<std::allocator<char>>(obj)->deallocate(ptr, n); }

std::allocator<char> a;
allocator_t zmqAlloc{
   &a,
   allocate,
   free_
};

void* ctx = zmq_ctx_new_with_allocator(&zmqAlloc);

I think this should work?

Best wishes,
Jens

--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
jens.auer at cgi.com
Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben.

CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.


> -----Original Message-----
> From: zeromq-dev [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of
> Luca Boccassi
> Sent: 28 November 2016 12:30
> To: ZeroMQ development list
> Subject: Re: [zeromq-dev] On hooking memory allocations
> 
> That would work for an internal API, but given we expose a C API unfortunately I
> don't think that would work as a public API :-( And I think for this use case they
> would require a public API.
> 
> As an external API, a new zmq_ctx_set that takes a callback would have been ideal,
> but it only takes int. So perhaps a new zmq_ctx_set_allocator that takes a callback
> pointer would be the next best.
> 
> An alternative would be to have a system similar to what we use for the poll
> implementation (epoll kqueue select etc), but this would be a build-time option,
> and the implementation would have to be checked in, which I don't think is an
> option for this case, right?
> 
> On Mon, 2016-11-28 at 10:51 +0000, Auer, Jens wrote:
> > Hi,
> >
> > I am just a user, but I would love to see this change. I have thinking
> > about this and I would like to be able to pass a C++ allocator object
> > to ZeroMQ, so a simple function hook is not enough. My idea is to
> > define a struct in the interface
> >
> > struct allocator_t
> > {
> >     void* hint;
> >     void* (allocate)(size_t, void*);
> >     void (deallocate)(void*, size_t, void*); };
> >
> > and store this in the context object. Since I don't think that this should be
> changed during runtime, I would create a new zmq_ctx_new overload which takes a
> parameter of type allocator_t. The default value would be to call malloc/free.
> >
> > Cheers,
> > Jens
> >
> > --
> > Jens Auer | CGI | Software-Engineer
> > CGI (Germany) GmbH & Co. KG
> > Rheinstraße 95 | 64295 Darmstadt | Germany
> > T: +49 6151 36860 154
> > jens.auer at cgi.com<mailto:jens.auer at cgi.com>
> > Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
> de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
> >
> > CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI
> Group Inc. and its affiliates may be contained in this message. If you are not a
> recipient indicated or intended in this message (or responsible for delivery of this
> message to such person), or you think for any reason that this message may have
> been addressed to you in error, you may not use or copy or deliver this message to
> anyone else. In such case, you should destroy this message and are asked to notify
> the sender by reply e-mail.
> > ________________________________
> > Von: zeromq-dev [zeromq-dev-bounces at lists.zeromq.org]" im Auftrag von
> > "Petteri Salo [petteri.salo at gmail.com]
> > Gesendet: Montag, 28. November 2016 09:40
> > An: zeromq-dev at lists.zeromq.org
> > Betreff: [zeromq-dev] On hooking memory allocations
> >
> > Hello,
> >
> > Let me first do a little introduction as I'm new to this list. I'm a software engineer
> with 15+ years of experience working on games at a company called Remedy
> Entertainment Ltd. We've done games for PC, and various games consoles over the
> years. Most recently we did Quantum Break for Xbox One.
> >
> > I've now been tasked with evaluating ZeroMQ. One important feature of any
> library we use in games is the ability to hook all memory allocations - this is to allow
> the use of custom memory allocators and/or for tracking when and where memory is
> allocated.
> >
> > I've searched the libzmq source code and there is about 150 uses of new, malloc,
> realloc , etc.
> >
> > If we were to adopt libzmq we'd like to put in allocation hooks and that work
> would then be something that we'd like to contribute back to the project. Having
> those hooks in the main repository would then make it easier for us to adopt future
> changes to the library.
> >
> > So, my question is would this kind of change be something that would be
> accepted? Of course assuming that coding conventions, proper way of submitting
> the patch etc. are followed. I do realize that one would want to see the actual code
> before accepting. I'm interested in the principle of accepting a change such as this,
> since it would introduce a new "rule" for working ión libzmq source code : "All
> allocations shall go through an allocation hook."
> >
> > Best Regards,
> >
> > Petteri Salo
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>