[zeromq-dev] On hooking memory allocations

[Petteri Salo]

Hi,

I haven't done enough analysis (yet) about current allocations in libzmq to
say which would work best for us, compile time or a runtime hooking. But
given that libzmq comes with source code, I would probably look for a
compile time option at first, like a macro. So roughly:

#ifdef USE_DEFAULT_ALLOCS
#define ZMQ_MALLOC ( ...  ) malloc( ... )
#define ZMQ_FREE ( ...  ) free( ... )
#else
#define ZMQ_MALLOC ( ...  ) my_custom_malloc( ... )
#define ZMQ_FREE ( ...  ) my_custom_free( ... )
#endif

and then use placement new in those places where the current code uses a
(plain) new.

-Petteri

On Mon, Nov 28, 2016 at 1:29 PM, Luca Boccassi <luca.boccassi at gmail.com>
wrote:

> That would work for an internal API, but given we expose a C API
> unfortunately I don't think that would work as a public API :-( And I
> think for this use case they would require a public API.
>
> As an external API, a new zmq_ctx_set that takes a callback would have
> been ideal, but it only takes int. So perhaps a new
> zmq_ctx_set_allocator that takes a callback pointer would be the next
> best.
>
> An alternative would be to have a system similar to what we use for the
> poll implementation (epoll kqueue select etc), but this would be a
> build-time option, and the implementation would have to be checked in,
> which I don't think is an option for this case, right?
>
> On Mon, 2016-11-28 at 10:51 +0000, Auer, Jens wrote:
> > Hi,
> >
> > I am just a user, but I would love to see this change. I have thinking
> about this and I would like to be able to pass a C++ allocator object to
> ZeroMQ, so a simple function hook is not enough. My idea is to define a
> struct in the interface
> >
> > struct allocator_t
> > {
> >     void* hint;
> >     void* (allocate)(size_t, void*);
> >     void (deallocate)(void*, size_t, void*);
> > };
> >
> > and store this in the context object. Since I don't think that this
> should be changed during runtime, I would create a new zmq_ctx_new overload
> which takes a parameter of type allocator_t. The default value would be to
> call malloc/free.
> >
> > Cheers,
> > Jens
> >
> > --
> > Jens Auer | CGI | Software-Engineer
> > CGI (Germany) GmbH & Co. KG
> > Rheinstraße 95 | 64295 Darmstadt | Germany
> > T: +49 6151 36860 154
> > jens.auer at cgi.com<mailto:jens.auer at cgi.com>
> > Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
> >
> > CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
> to CGI Group Inc. and its affiliates may be contained in this message. If
> you are not a recipient indicated or intended in this message (or
> responsible for delivery of this message to such person), or you think for
> any reason that this message may have been addressed to you in error, you
> may not use or copy or deliver this message to anyone else. In such case,
> you should destroy this message and are asked to notify the sender by reply
> e-mail.
> > ________________________________
> > Von: zeromq-dev [zeromq-dev-bounces at lists.zeromq.org]" im Auftrag von
> "Petteri Salo [petteri.salo at gmail.com]
> > Gesendet: Montag, 28. November 2016 09:40
> > An: zeromq-dev at lists.zeromq.org
> > Betreff: [zeromq-dev] On hooking memory allocations
> >
> > Hello,
> >
> > Let me first do a little introduction as I'm new to this list. I'm a
> software engineer with 15+ years of experience working on games at a
> company called Remedy Entertainment Ltd. We've done games for PC, and
> various games consoles over the years. Most recently we did Quantum Break
> for Xbox One.
> >
> > I've now been tasked with evaluating ZeroMQ. One important feature of
> any library we use in games is the ability to hook all memory allocations -
> this is to allow the use of custom memory allocators and/or for tracking
> when and where memory is allocated.
> >
> > I've searched the libzmq source code and there is about 150 uses of new,
> malloc, realloc , etc.
> >
> > If we were to adopt libzmq we'd like to put in allocation hooks and that
> work would then be something that we'd like to contribute back to the
> project. Having those hooks in the main repository would then make it
> easier for us to adopt future changes to the library.
> >
> > So, my question is would this kind of change be something that would be
> accepted? Of course assuming that coding conventions, proper way of
> submitting the patch etc. are followed. I do realize that one would want to
> see the actual code before accepting. I'm interested in the principle of
> accepting a change such as this, since it would introduce a new "rule" for
> working ión libzmq source code : "All allocations shall go through an
> allocation hook."
> >
> > Best Regards,
> >
> > Petteri Salo
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20161128/407d4c5c/attachment.htm>