[zeromq-dev] (almost) zero-copy message receive

Auer, Jens jens.auer at cgi.com
Tue Jun 2 16:40:07 CEST 2015


Hi Thomas,

I did not want to say that you intended to start a flame war and you certainly did not. I know that things like these easily turn into flame wars, so I wanted to prevent that. All I wanted say was that I know that switching to C++11 may not be applicable so I want to discuss the patch first and search for an alternative to C++11. Do you have a suggestion for the issue with the non-POD atomic_counter_t? My experience is quite limited because I normally don’t write code which has to support older compilers.

Just out of curiosity, do you know a definite list of compilers zeroMQ intends to supports? I only found the document http://zeromq.org/docs:builds, but this is quite outdated.

Best wishes,
  Jens

--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
jens.auer at cgi.com<mailto:jens.auer at cgi.com>
Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.

CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.

From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Thomas Rodgers
Sent: 02 June 2015 16:07
To: ZeroMQ development list
Subject: Re: [zeromq-dev] (almost) zero-copy message receive

A better chart of compiler conformance is probably here -

http://en.cppreference.com/w/cpp/compiler_support

I don't know that there's been a "flamewar" around this topic, but switching to -std=c++11 has been discussed recently in the context of implementing thread safe sockets. IIRC the consensus at the time was to stick with 98/03 conforming code.

In my experience, supporting C++11, opens opens up a bit of a minefield of portability "gotchas". For https://github.com/zeromq/azmq I can generally fall back on Boost to paper over these issues, but that's not a realistic option with libzmq. So, while I get that *this* change wouldn't necessarily run afoul of the current state of C++ conformance for MSVC, I am personally a bit leary of the general prospects of switching to C++11 in libzmq because the next change that assumes C++11 conformance might not be supported by the range of compilers used to build libzmq.



On Tue, Jun 2, 2015 at 8:44 AM, Auer, Jens <jens.auer at cgi.com<mailto:jens.auer at cgi.com>> wrote:
Hi,

I don't want to start a flame war on this because I know that there can be very good reasons to not rely on C++11. That's why I did not just send the patch, but wanted to discuss it first and see if there are any other ways to do this.
The main issue is that I want to put an atomic_counter_t into a union in msg_t, which C++98/C++03 does not allow because it has constructors and non-public data members. So an obvious way to solve this would be to make atomic_counter_t a plain C struct with free function to create and modify it. I was hoping for other suggestions.

For what it's worth, the change does not need a fully compliant C++, but just standard layout and trivial types which are implemented in MSVC since 2012.

Cheers,
  Jens

--
Jens Auer | CGI | Software-Engineer
CGI (Germany) GmbH & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154<tel:%2B49%206151%2036860%20154>
jens.auer at cgi.com<mailto:jens.auer at cgi.com>
Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.

CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.
________________________________
Von: zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org> [zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org>]" im Auftrag von "Thomas Rodgers [rodgert at twrodgers.com<mailto:rodgert at twrodgers.com>]
Gesendet: Dienstag, 2. Juni 2015 15:37

An: ZeroMQ development list
Betreff: Re: [zeromq-dev] (almost) zero-copy message receive

Personally, I think that 4 years of C++11, this should not be an issues, but there may be platforms with old compilers which you want to support.

4 years of C++11 *should* be enough, but wide-spread use of fully conforming compilers is still an issue, for instance -

https://msdn.microsoft.com/en-us/library/hh567368.aspx

On Tue, Jun 2, 2015 at 8:05 AM, Auer, Jens <jens.auer at cgi.com<mailto:jens.auer at cgi.com>> wrote:
Hi Pieter,

the reason I wanted to ask first is because I had to switch on C++11 to make it work without changing atomic_counter_t. The reason is that I eliminated msg_t::content_t completely to save a mallic call  by adding the members in content_t to the msg_t class directly since there is now space enough. However, atomic_counter_t is not a POD and cannot be put into the union. For my proof-of-concept, switching on C++11 is fine, but I am not sure if that is ok for the main branch. Personally, I think that 4 years of C++11, this should not be an issues, but there may be platforms with old compilers which you want to support.

The only alternative I came up with would be to make atomic_counter_t a classical C struct with free functions instead of a class. I don't like this very much.

Best wishes,
  Jens

--
Jens Auer | CGI | Software-Engineer
CGI (Germany) GmbH & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154<tel:%2B49%206151%2036860%20154>
jens.auer at cgi.com<mailto:jens.auer at cgi.com>
Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.

CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.

________________________________________
Von: zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org> [zeromq-dev-bounces at lists.zeromq.org<mailto:zeromq-dev-bounces at lists.zeromq.org>]" im Auftrag von "Pieter Hintjens [ph at imatix.com<mailto:ph at imatix.com>]
Gesendet: Dienstag, 2. Juni 2015 10:18
An: ZeroMQ development list
Betreff: Re: [zeromq-dev] (almost) zero-copy message receive
Jens,

Sounds great. Feel free to send such patches to libzmq master; please
make sure they are as atomic as possible, each with a clear problem
statement, each testable individually.

-Pieter

On Tue, Jun 2, 2015 at 10:13 AM, Arnaud Loonstra <arnaud at sphaero.org<mailto:arnaud at sphaero.org>> wrote:
> Although I'm not very familiar with zmq's internals this looks
> promising.
> Did you test if your implementation remains correct? ie. it doesn't
> introduce deadlocks or other race conditions?
>
> Rg,
>
> Arnaud
>
> On 2015-05-31 19:29, Jens Auer wrote:
>> Hi,
>>
>> I did some performance analysis of  a program which receives data on
>> a (SUB or
>> PULL) socket, filters it for some criteria, extracts a value from the
>> message
>> and uses this as a subscription to forward the datato a PUB socket.
>> As
>> expected, most time is spent in memory allocations and memcpy
>> operations, so I
>> decided to check if there is an opportunity to  minimize these
>> operations in
>> the critical path. From my analysis, the path is as follows:
>> 1. stream_engine receives data from a socket into a static buffer of
>> 8192
>> bytes
>> 2. decoder/v2_decoder implement a state machine which reads the flag
>> and
>> message size, create a new message and copy the data into the message
>> data
>> field
>> 3. When sending, stream_engine copies the flags field, message and
>> message
>> data into a static buffer and sends this buffer completely to the
>> socket
>>
>> Memory allocations are done in v2_decoder when a new message is
>> created, and
>> deallocations are done when sending the message. Memcpy operations
>> are done in
>> decoder to copy
>> - the flags byte into a temporary buffer
>> - the message size into a temporary buffer
>> - the message data into the dynamically allocated storage
>>
>> Since the allocations and memcpy are the dominating operations, I
>> implemented
>> a scheme where these operations are minimized. The main idea is to
>> allocate
>> the receive buffer of 8192 byte dynamically and use this as the data
>> storage
>> for zero-copy messages created with msg_t::init_data. This replaces n
>> = 8192 /
>> (m_size + 10) memory allocations with one allocation, and it gets rid
>> of the
>> same number of memcpy operations for the message data. I implemented
>> this in a
>> fork (https://github.com/jens-auer/libzmq/tree/zero_copy_receive).
>> For
>> testing, I ran the throughput test (message size 100, 100000
>> messages) locally
>> and profiled for memory allocations and memcpy. The results are
>> promising:
>> - memory allocations reduced from 100,260 to 2,573
>> - memcpy operations reduced from 301,227 to 202,449. This is expected
>> because
>> for every message, three memcpys are done, and the patch removes the
>> data
>> memcpy only.
>> - throughput increased significantly by about 30-40% ( I only did a
>> couple of
>> runs to test it, no thorough benchmarking)
>>
>> For the implementation, I had to change two other things. After my
>> first
>> implementation, I realized that msg_t::init_data does a malloc to
>> create the
>> content_t member. Given that msg_t's size is now 64 bytes, I removed
>> content_t
>> completely by adding the members of content_t to the lmsg_t union.
>> However,
>> this is problem with the current code because one of the members is a
>> atomic_counter_t which is a non-POD type and cannot be a union
>> member. For my
>> proof-of-concept implementation, I switched on C++11 mode because
>> this relaxes
>> the requirements for PODs.
>>
>> I hope this could be useful and maybe included in the main branch. My
>> next
>> step is to change the encoder/stream engine to use writev to skip the
>> memcpy
>> operations when sending messages.
>>
>> Best wishes,
>>   Jens Auer
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150602/135f0837/attachment.htm>


More information about the zeromq-dev mailing list