[zeromq-dev] (almost) zero-copy message receive

[Thomas Rodgers]

I am unaware of any list, it seems (to me at least) it's whatever people
have contributed support for.

As for a concrete suggestion. I am not sure this is any less ugly but...

libzmq goes out of it's way generally to avoid calling new() in many cases
(non-throwing failure on OOM presumably), preferring malloc() and placement
new instead, content_t was no exception. It did embed an explicit
atomic_counter_t and would placement new it, and explicitly call the
destructor when the lmsg was destroyed.

You could, instead declare the storage for atomic_counter_t as an array,
uint8_t ctr_storage[sizeof(atomic_counter_t)] and placement new/explicit
dtor as before, and then provide an accessor to return an atomic_counter_t&
by returning *reinterpret_cast<atomic_counter_t*>(&ctr_storage[0]).

It doesn't change the size of the lmsg type, but it does remove one level
of indirection, and you have hidden the ugly bits behind an accessor method.

On Tue, Jun 2, 2015 at 9:40 AM, Auer, Jens <jens.auer at cgi.com> wrote:

>  Hi Thomas,
>
>
>
> I did not want to say that you intended to start a flame war and you
> certainly did not. I know that things like these easily turn into flame
> wars, so I wanted to prevent that. All I wanted say was that I know that
> switching to C++11 may not be applicable so I want to discuss the patch
> first and search for an alternative to C++11. Do you have a suggestion for
> the issue with the non-POD atomic_counter_t? My experience is quite
> limited because I normally don’t write code which has to support older
> compilers.
>
>
>
> Just out of curiosity, do you know a definite list of compilers zeroMQ
> intends to supports? I only found the document
> http://zeromq.org/docs:builds, but this is quite outdated.
>
>
>
> Best wishes,
>
>   Jens
>
>
>
> *--*
>
> *Dr. Jens Auer *| CGI | Software Engineer
>
> CGI Deutschland Ltd. & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
>
> T: +49 6151 36860 154
>
> jens.auer at cgi.com
>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben.
>
>
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
>
>
> *From:* zeromq-dev-bounces at lists.zeromq.org [mailto:
> zeromq-dev-bounces at lists.zeromq.org] *On Behalf Of *Thomas Rodgers
> *Sent:* 02 June 2015 16:07
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] (almost) zero-copy message receive
>
>
>
> A better chart of compiler conformance is probably here -
>
>
>
> http://en.cppreference.com/w/cpp/compiler_support
>
>
>
> I don't know that there's been a "flamewar" around this topic, but
> switching to -std=c++11 has been discussed recently in the context of
> implementing thread safe sockets. IIRC the consensus at the time was to
> stick with 98/03 conforming code.
>
>
>
> In my experience, supporting C++11, opens opens up a bit of a minefield of
> portability "gotchas". For https://github.com/zeromq/azmq I can generally
> fall back on Boost to paper over these issues, but that's not a realistic
> option with libzmq. So, while I get that *this* change wouldn't necessarily
> run afoul of the current state of C++ conformance for MSVC, I am personally
> a bit leary of the general prospects of switching to C++11 in libzmq
> because the next change that assumes C++11 conformance might not be
> supported by the range of compilers used to build libzmq.
>
>
>
>
>
>
>
> On Tue, Jun 2, 2015 at 8:44 AM, Auer, Jens <jens.auer at cgi.com> wrote:
>
> Hi,
>
> I don't want to start a flame war on this because I know that there can be
> very good reasons to not rely on C++11. That's why I did not just send the
> patch, but wanted to discuss it first and see if there are any other ways
> to do this.
> The main issue is that I want to put an atomic_counter_t into a union in
> msg_t, which C++98/C++03 does not allow because it has constructors and
> non-public data members. So an obvious way to solve this would be to make
> atomic_counter_t a plain C struct with free function to create and modify
> it. I was hoping for other suggestions.
>
> For what it's worth, the change does not need a fully compliant C++, but
> just standard layout and trivial types which are implemented in MSVC since
> 2012.
>
> Cheers,
>   Jens
>
>
>
> --
>
> *Jens Auer *| CGI | Software-Engineer
>
> CGI (Germany) GmbH & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
>
> T: +49 6151 36860 154
> jens.auer at cgi.com
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
> de.cgi.com/pflichtangaben.
>
>
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>         ------------------------------
>
> *Von:* zeromq-dev-bounces at lists.zeromq.org [
> zeromq-dev-bounces at lists.zeromq.org]" im Auftrag von "Thomas Rodgers [
> rodgert at twrodgers.com]
> *Gesendet:* Dienstag, 2. Juni 2015 15:37
>
>
> *An:* ZeroMQ development list
> *Betreff:* Re: [zeromq-dev] (almost) zero-copy message receive
>
>
>
> Personally, I think that 4 years of C++11, this should not be an issues,
> but there may be platforms with old compilers which you want to support.
>
>
>
> 4 years of C++11 *should* be enough, but wide-spread use of fully
> conforming compilers is still an issue, for instance -
>
>
>
> https://msdn.microsoft.com/en-us/library/hh567368.aspx
>
>
>
> On Tue, Jun 2, 2015 at 8:05 AM, Auer, Jens <jens.auer at cgi.com> wrote:
>
> Hi Pieter,
>
> the reason I wanted to ask first is because I had to switch on C++11 to
> make it work without changing atomic_counter_t. The reason is that I
> eliminated msg_t::content_t completely to save a mallic call  by adding the
> members in content_t to the msg_t class directly since there is now space
> enough. However, atomic_counter_t is not a POD and cannot be put into the
> union. For my proof-of-concept, switching on C++11 is fine, but I am not
> sure if that is ok for the main branch. Personally, I think that 4 years of
> C++11, this should not be an issues, but there may be platforms with old
> compilers which you want to support.
>
> The only alternative I came up with would be to make atomic_counter_t a
> classical C struct with free functions instead of a class. I don't like
> this very much.
>
> Best wishes,
>   Jens
>
> --
> Jens Auer | CGI | Software-Engineer
> CGI (Germany) GmbH & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
> T: +49 6151 36860 154
> jens.auer at cgi.com
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben.
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
> ________________________________________
> Von: zeromq-dev-bounces at lists.zeromq.org [
> zeromq-dev-bounces at lists.zeromq.org]" im Auftrag von "Pieter
> Hintjens [ph at imatix.com]
> Gesendet: Dienstag, 2. Juni 2015 10:18
> An: ZeroMQ development list
> Betreff: Re: [zeromq-dev] (almost) zero-copy message receive
>
> Jens,
>
> Sounds great. Feel free to send such patches to libzmq master; please
> make sure they are as atomic as possible, each with a clear problem
> statement, each testable individually.
>
> -Pieter
>
> On Tue, Jun 2, 2015 at 10:13 AM, Arnaud Loonstra <arnaud at sphaero.org>
> wrote:
> > Although I'm not very familiar with zmq's internals this looks
> > promising.
> > Did you test if your implementation remains correct? ie. it doesn't
> > introduce deadlocks or other race conditions?
> >
> > Rg,
> >
> > Arnaud
> >
> > On 2015-05-31 19:29, Jens Auer wrote:
> >> Hi,
> >>
> >> I did some performance analysis of  a program which receives data on
> >> a (SUB or
> >> PULL) socket, filters it for some criteria, extracts a value from the
> >> message
> >> and uses this as a subscription to forward the datato a PUB socket.
> >> As
> >> expected, most time is spent in memory allocations and memcpy
> >> operations, so I
> >> decided to check if there is an opportunity to  minimize these
> >> operations in
> >> the critical path. From my analysis, the path is as follows:
> >> 1. stream_engine receives data from a socket into a static buffer of
> >> 8192
> >> bytes
> >> 2. decoder/v2_decoder implement a state machine which reads the flag
> >> and
> >> message size, create a new message and copy the data into the message
> >> data
> >> field
> >> 3. When sending, stream_engine copies the flags field, message and
> >> message
> >> data into a static buffer and sends this buffer completely to the
> >> socket
> >>
> >> Memory allocations are done in v2_decoder when a new message is
> >> created, and
> >> deallocations are done when sending the message. Memcpy operations
> >> are done in
> >> decoder to copy
> >> - the flags byte into a temporary buffer
> >> - the message size into a temporary buffer
> >> - the message data into the dynamically allocated storage
> >>
> >> Since the allocations and memcpy are the dominating operations, I
> >> implemented
> >> a scheme where these operations are minimized. The main idea is to
> >> allocate
> >> the receive buffer of 8192 byte dynamically and use this as the data
> >> storage
> >> for zero-copy messages created with msg_t::init_data. This replaces n
> >> = 8192 /
> >> (m_size + 10) memory allocations with one allocation, and it gets rid
> >> of the
> >> same number of memcpy operations for the message data. I implemented
> >> this in a
> >> fork (https://github.com/jens-auer/libzmq/tree/zero_copy_receive).
> >> For
> >> testing, I ran the throughput test (message size 100, 100000
> >> messages) locally
> >> and profiled for memory allocations and memcpy. The results are
> >> promising:
> >> - memory allocations reduced from 100,260 to 2,573
> >> - memcpy operations reduced from 301,227 to 202,449. This is expected
> >> because
> >> for every message, three memcpys are done, and the patch removes the
> >> data
> >> memcpy only.
> >> - throughput increased significantly by about 30-40% ( I only did a
> >> couple of
> >> runs to test it, no thorough benchmarking)
> >>
> >> For the implementation, I had to change two other things. After my
> >> first
> >> implementation, I realized that msg_t::init_data does a malloc to
> >> create the
> >> content_t member. Given that msg_t's size is now 64 bytes, I removed
> >> content_t
> >> completely by adding the members of content_t to the lmsg_t union.
> >> However,
> >> this is problem with the current code because one of the members is a
> >> atomic_counter_t which is a non-POD type and cannot be a union
> >> member. For my
> >> proof-of-concept implementation, I switched on C++11 mode because
> >> this relaxes
> >> the requirements for PODs.
> >>
> >> I hope this could be useful and maybe included in the main branch. My
> >> next
> >> step is to change the encoder/stream engine to use writev to skip the
> >> memcpy
> >> operations when sending messages.
> >>
> >> Best wishes,
> >>   Jens Auer
> >>
> >>
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150602/723ce37d/attachment.htm>