[zeromq-dev] (almost) zero-copy message receive

Thomas Rodgers rodgert at twrodgers.com
Tue Jun 2 19:14:37 CEST 2015 

Current vector implementations (AFAIK) don't do small object optimizations,
but I have it on good authority (the author sits next to me at work) there
is a proposal coming for such a vector, and it would face the exact same
problem. std::vector does use a dynamically allocated array, but not of
type T, otherwise reserve() would unimplementable. Is the point that a
void* that comes via malloc is somehow different than a member array in
terms of type punning? This may well be strictly true (see aforementioned
malloc "standards weasel wording" comment), but there are many many
libraries which do small object optimizations via the double static_cast
approach that would break if it didn't work (never mind whether it is
legal). The standard already includes types that do this sort of punning
(std::function, std::aligned_storage, proposed std::optional, proposed
std::variant).

In any event, I've dropped an email to a couple of fellow C++ standards
committee members (including a Clang developer) to see what their thoughts
on this are.



On Tue, Jun 2, 2015 at 11:53 AM, Jens Auer <jens.auer at betaversion.net>
wrote:

> I always thought that vector uses a dynamically allocated array and not a
> char []. That would give us a char* which we can savely cast into another
> pointer. I don't have the source available, but if it uses small object
> optimization, it has to use a static array backend. I will take a look at
> the vector source code.
>
> Thanks,
>  Jens
>
>
>
> -------- Ursprüngliche Nachricht --------
> Von: Thomas Rodgers <rodgert at twrodgers.com>
> Datum: 02.06.2015 18:37 (GMT+01:00)
> An: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
> Betreff: Re: [zeromq-dev] (almost) zero-copy message receive
>
>
> If it were completely broken, then how could vector work (it is the
> current poster child for type erased aligned storage)? std::aligned_storage
> (and boost::aligned_storage) also do type erasure through a char backing
> array. It might well be "strictly illegal" relative to standards wording
> (there was some discussion along these lines at the fall Standards
> meeting), but it would essentially preclude any sort of placement new
> except the via the "magic weasel" words that allow malloc.
>
> On Tue, Jun 2, 2015 at 11:30 AM, Auer, Jens <jens.auer at cgi.com> wrote:
>
>>  Yes, this doesn't change anything because it only depends on the
>> existence of two pointers of different type pointing to the same address.
>> There is an exception for char*, but here we would have a char array and an
>> atomic_counter_t pointer pointing to it. This is illegal.
>>
>> Cheers,
>>   Jens
>>
>>      --
>>  *Jens Auer *| CGI | Software-Engineer
>>  CGI (Germany) GmbH & Co. KG
>> Rheinstraße 95 | 64295 Darmstadt | Germany
>> T: +49 6151 36860 154
>> *jens.auer at cgi.com* <jens.auer at cgi.com>
>> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
>> *de.cgi.com/pflichtangaben* <http://de.cgi.com/pflichtangaben>.
>>
>> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
>> CGI Group Inc. and its affiliates may be contained in this message. If you
>> are not a recipient indicated or intended in this message (or responsible
>> for delivery of this message to such person), or you think for any reason
>> that this message may have been addressed to you in error, you may not use
>> or copy or deliver this message to anyone else. In such case, you should
>> destroy this message and are asked to notify the sender by reply e-mail.
>>       ------------------------------
>> *Von:* zeromq-dev-bounces at lists.zeromq.org [
>> zeromq-dev-bounces at lists.zeromq.org]" im Auftrag von "Thomas Rodgers [
>> rodgert at twrodgers.com]
>> *Gesendet:* Dienstag, 2. Juni 2015 18:10
>>
>> *An:* ZeroMQ development list
>> *Betreff:* Re: [zeromq-dev] (almost) zero-copy message receive
>>
>>   with reinterpret_cast?
>>
>> On Tue, Jun 2, 2015 at 10:55 AM, Auer, Jens <jens.auer at cgi.com> wrote:
>>
>>>  I already tried this, but then the compiler complaint about a
>>> violation of the strict alias rules and the code doesn't compile anymore.
>>> The problem is that I access something of type uint8_t[] with a pointer of
>>> type atomic_counter_t.
>>>
>>> Cheers,
>>>   Jens
>>>
>>>      --
>>>  *Jens Auer *| CGI | Software-Engineer
>>>  CGI (Germany) GmbH & Co. KG
>>> Rheinstraße 95 | 64295 Darmstadt | Germany
>>> T: +49 6151 36860 154
>>> *jens.auer at cgi.com* <jens.auer at cgi.com>
>>> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
>>> *de.cgi.com/pflichtangaben* <http://de.cgi.com/pflichtangaben>.
>>>
>>> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
>>> to CGI Group Inc. and its affiliates may be contained in this message. If
>>> you are not a recipient indicated or intended in this message (or
>>> responsible for delivery of this message to such person), or you think for
>>> any reason that this message may have been addressed to you in error, you
>>> may not use or copy or deliver this message to anyone else. In such case,
>>> you should destroy this message and are asked to notify the sender by reply
>>> e-mail.
>>>        ------------------------------
>>> *Von:* zeromq-dev-bounces at lists.zeromq.org [
>>> zeromq-dev-bounces at lists.zeromq.org]" im Auftrag von "Thomas Rodgers [
>>> rodgert at twrodgers.com]
>>> *Gesendet:* Dienstag, 2. Juni 2015 17:34
>>>
>>> *An:* ZeroMQ development list
>>> *Betreff:* Re: [zeromq-dev] (almost) zero-copy message receive
>>>
>>>    One other point, you would need to guarantee appropriate alignment
>>> of the backing array (sadly, C++11 gave us std::aligned_storage but 03 not
>>> so much).
>>>
>>> On Tue, Jun 2, 2015 at 10:11 AM, Thomas Rodgers <rodgert at twrodgers.com>
>>> wrote:
>>>
>>>> I am unaware of any list, it seems (to me at least) it's whatever
>>>> people have contributed support for.
>>>>
>>>>  As for a concrete suggestion. I am not sure this is any less ugly
>>>> but...
>>>>
>>>>  libzmq goes out of it's way generally to avoid calling new() in many
>>>> cases (non-throwing failure on OOM presumably), preferring malloc() and
>>>> placement new instead, content_t was no exception. It did embed an explicit
>>>> atomic_counter_t and would placement new it, and explicitly call the
>>>> destructor when the lmsg was destroyed.
>>>>
>>>>  You could, instead declare the storage for atomic_counter_t as an
>>>> array, uint8_t ctr_storage[sizeof(atomic_counter_t)] and placement
>>>> new/explicit dtor as before, and then provide an accessor to return an
>>>> atomic_counter_t& by returning
>>>> *reinterpret_cast<atomic_counter_t*>(&ctr_storage[0]).
>>>>
>>>>  It doesn't change the size of the lmsg type, but it does remove one
>>>> level of indirection, and you have hidden the ugly bits behind an accessor
>>>> method.
>>>>
>>>> On Tue, Jun 2, 2015 at 9:40 AM, Auer, Jens <jens.auer at cgi.com> wrote:
>>>>
>>>>>  Hi Thomas,
>>>>>
>>>>>
>>>>>
>>>>> I did not want to say that you intended to start a flame war and you
>>>>> certainly did not. I know that things like these easily turn into flame
>>>>> wars, so I wanted to prevent that. All I wanted say was that I know that
>>>>> switching to C++11 may not be applicable so I want to discuss the patch
>>>>> first and search for an alternative to C++11. Do you have a suggestion for
>>>>> the issue with the non-POD atomic_counter_t? My experience is quite
>>>>> limited because I normally don’t write code which has to support older
>>>>> compilers.
>>>>>
>>>>>
>>>>>
>>>>> Just out of curiosity, do you know a definite list of compilers zeroMQ
>>>>> intends to supports? I only found the document
>>>>> http://zeromq.org/docs:builds, but this is quite outdated.
>>>>>
>>>>>
>>>>>
>>>>> Best wishes,
>>>>>
>>>>>   Jens
>>>>>
>>>>>
>>>>>
>>>>> *--*
>>>>>
>>>>> *Dr. Jens Auer *| CGI | Software Engineer
>>>>>
>>>>> CGI Deutschland Ltd. & Co. KG
>>>>> Rheinstraße 95 | 64295 Darmstadt | Germany
>>>>>
>>>>> T: +49 6151 36860 154
>>>>>
>>>>> jens.auer at cgi.com
>>>>>
>>>>> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
>>>>> unter de.cgi.com/pflichtangaben.
>>>>>
>>>>>
>>>>>
>>>>> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
>>>>> to CGI Group Inc. and its affiliates may be contained in this message. If
>>>>> you are not a recipient indicated or intended in this message (or
>>>>> responsible for delivery of this message to such person), or you think for
>>>>> any reason that this message may have been addressed to you in error, you
>>>>> may not use or copy or deliver this message to anyone else. In such case,
>>>>> you should destroy this message and are asked to notify the sender by reply
>>>>> e-mail.
>>>>>
>>>>>
>>>>>
>>>>> *From:* zeromq-dev-bounces at lists.zeromq.org [mailto:
>>>>> zeromq-dev-bounces at lists.zeromq.org] *On Behalf Of *Thomas Rodgers
>>>>> *Sent:* 02 June 2015 16:07
>>>>> *To:* ZeroMQ development list
>>>>> *Subject:* Re: [zeromq-dev] (almost) zero-copy message receive
>>>>>
>>>>>
>>>>>
>>>>> A better chart of compiler conformance is probably here -
>>>>>
>>>>>
>>>>>
>>>>> http://en.cppreference.com/w/cpp/compiler_support
>>>>>
>>>>>
>>>>>
>>>>> I don't know that there's been a "flamewar" around this topic, but
>>>>> switching to -std=c++11 has been discussed recently in the context of
>>>>> implementing thread safe sockets. IIRC the consensus at the time was to
>>>>> stick with 98/03 conforming code.
>>>>>
>>>>>
>>>>>
>>>>> In my experience, supporting C++11, opens opens up a bit of a
>>>>> minefield of portability "gotchas". For https://github.com/zeromq/azmq
>>>>> I can generally fall back on Boost to paper over these issues, but that's
>>>>> not a realistic option with libzmq. So, while I get that *this* change
>>>>> wouldn't necessarily run afoul of the current state of C++ conformance for
>>>>> MSVC, I am personally a bit leary of the general prospects of switching to
>>>>> C++11 in libzmq because the next change that assumes C++11 conformance
>>>>> might not be supported by the range of compilers used to build libzmq.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 2, 2015 at 8:44 AM, Auer, Jens <jens.auer at cgi.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I don't want to start a flame war on this because I know that there
>>>>> can be very good reasons to not rely on C++11. That's why I did not just
>>>>> send the patch, but wanted to discuss it first and see if there are any
>>>>> other ways to do this.
>>>>> The main issue is that I want to put an atomic_counter_t into a union
>>>>> in msg_t, which C++98/C++03 does not allow because it has constructors and
>>>>> non-public data members. So an obvious way to solve this would be to make
>>>>> atomic_counter_t a plain C struct with free function to create and modify
>>>>> it. I was hoping for other suggestions.
>>>>>
>>>>> For what it's worth, the change does not need a fully compliant C++,
>>>>> but just standard layout and trivial types which are implemented in MSVC
>>>>> since 2012.
>>>>>
>>>>> Cheers,
>>>>>   Jens
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Jens Auer *| CGI | Software-Engineer
>>>>>
>>>>> CGI (Germany) GmbH & Co. KG
>>>>> Rheinstraße 95 | 64295 Darmstadt | Germany
>>>>>
>>>>> T: +49 6151 36860 154
>>>>> jens.auer at cgi.com
>>>>> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
>>>>> unter de.cgi.com/pflichtangaben.
>>>>>
>>>>>
>>>>>
>>>>> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
>>>>> to CGI Group Inc. and its affiliates may be contained in this message. If
>>>>> you are not a recipient indicated or intended in this message (or
>>>>> responsible for delivery of this message to such person), or you think for
>>>>> any reason that this message may have been addressed to you in error, you
>>>>> may not use or copy or deliver this message to anyone else. In such case,
>>>>> you should destroy this message and are asked to notify the sender by reply
>>>>> e-mail.
>>>>>         ------------------------------
>>>>>
>>>>> *Von:* zeromq-dev-bounces at lists.zeromq.org [
>>>>> zeromq-dev-bounces at lists.zeromq.org]" im Auftrag von "Thomas Rodgers [
>>>>> rodgert at twrodgers.com]
>>>>> *Gesendet:* Dienstag, 2. Juni 2015 15:37
>>>>>
>>>>>
>>>>> *An:* ZeroMQ development list
>>>>> *Betreff:* Re: [zeromq-dev] (almost) zero-copy message receive
>>>>>
>>>>>
>>>>>
>>>>> Personally, I think that 4 years of C++11, this should not be an
>>>>> issues, but there may be platforms with old compilers which you want to
>>>>> support.
>>>>>
>>>>>
>>>>>
>>>>> 4 years of C++11 *should* be enough, but wide-spread use of fully
>>>>> conforming compilers is still an issue, for instance -
>>>>>
>>>>>
>>>>>
>>>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150602/0ee10610/attachment.htm>

More information about the zeromq-dev mailing list