[zeromq-dev] New option to set receive buffer for tcp

Doron Somech somdoron at gmail.com
Thu Sep 24 19:36:26 CEST 2015


Increasing the buffer will not change the number of allocations, only the
number of recv API calls, I think this is the performance gain you see. The
buffer size is per one TCP connection, how much messages do you expect to
get on one TCP connection? I don't believe you will see performance
improvement on real world scenario ( multiple clients on remote computers).
Increasing the buffer can cause some problems like starvation and High CPU
(for copy operation).
On Sep 24, 2015 1:40 AM, "Auer, Jens" <jens.auer at cgi.com> wrote:

> Hi Pieter,
>
> I have still been thinking about the socket option because I think a good
> setting depends on properties only the user knows (i.e. message rate and
> average size). What would you think about an option with possible values
> 0...10 instead of symbolic names. We could set the buffer to 2^10kb, and
> the default would be 3. In the option documentation, it would just be
> stated that the values ranges from very small=0 to large=10, and the
> implementation is independent from the values.
>
> Best wishes,
>   Jens
>
> --
> Dr. Jens Auer | CGI | Software Engineer
> CGI Deutschland Ltd. & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
> T: +49 6151 36860 154
> jens.auer at cgi.com
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter de.cgi.com/pflichtangaben.
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
>
> > -----Original Message-----
> > From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-
> > bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
> > Sent: 14 July 2015 11:28
> > To: ZeroMQ development list
> > Subject: Re: [zeromq-dev] New option to set receive buffer for tcp
> >
> > Indeed, some environments can't use heap memory, so custom allocators
> into pre-
> > allocated memory could be helpful. One thing at a time though.
> >
> > For FAST I would assume that after say 1 Mb there is no measurable
> benefit in
> > increasing block size, no matter the rates and message sizes. It's
> probably lower
> > than that.
> >
> > On Tue, Jul 14, 2015 at 9:11 AM, Auer, Jens <jens.auer at cgi.com> wrote:
> > > Hi Pieter,
> > >
> > > How would we come up with a value for FAST? As I see it, the value
> > > depends on
> > > - the data rate (msg/s)
> > > - the message size
> > > - the amount of memory the developer wants to spend
> > >
> > > We can define a small struct as parameter to the new socket option
> with data
> > rate, message size and the maximum memory available to zeroMQ, and then
> decide
> > internally how much space we allocate. But I still it is hard to come up
> with a
> > general rule.
> > >
> > > I also thought about new socket options to set allocation/deallocation
> functions.
> > This would allow users to use a memory pool for the buffers, but it is a
> very risky
> > option with a lot of potential for problems. On the other hand, many
> people working
> > on systems with resource constraints would love it. What is your opinion
> on this?
> > >
> > > Best wishes,
> > >   Jens
> > >
> > > --
> > > Dr. Jens Auer | CGI | Software Engineer CGI Deutschland Ltd. & Co. KG
> > > Rheinstraße 95 | 64295 Darmstadt | Germany
> > > T: +49 6151 36860 154
> > > jens.auer at cgi.com
> > > Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter
> > de.cgi.com/pflichtangaben.
> > >
> > > CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
> to CGI
> > Group Inc. and its affiliates may be contained in this message. If you
> are not a
> > recipient indicated or intended in this message (or responsible for
> delivery of this
> > message to such person), or you think for any reason that this message
> may have
> > been addressed to you in error, you may not use or copy or deliver this
> message to
> > anyone else. In such case, you should destroy this message and are asked
> to notify
> > the sender by reply e-mail.
> > >
> > >
> > > -----Original Message-----
> > > From: zeromq-dev-bounces at lists.zeromq.org
> > > [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter
> > > Hintjens
> > > Sent: 13 July 2015 23:44
> > > To: ZeroMQ development list
> > > Subject: Re: [zeromq-dev] New option to set receive buffer for tcp
> > >
> > > A socket option is a good idea. Is there any way we can abstract this
> so users
> > don't need to calculate buffer sizes (they will get it wrong often
> enough)? E.g.
> > _SMALL vs. _FAST?
> > >
> > > On Mon, Jul 13, 2015 at 5:29 PM, Auer, Jens <jens.auer at cgi.com> wrote:
> > >> Hi,
> > >>
> > >>
> > >>
> > >> I ran some benchmarks with the current master to see if the zero-copy
> > >> changes have an effect on the performance in my system. As it turns
> > >> out, in my setup, the effect is not so large because of the number of
> > >> malloc calls still being done. In my setup, I send and receive data
> > >> streams with up to
> > >> 200000 messages per second with a message size of 1125 bytes. The
> > >> receive buffer is fixed to 8kb, so I need one allocation for every 7
> > >> messages, and one deallocation. I experimentally increased the size
> > >> of the receive buffer to 65kb, and this resulted in a good
> > >> performance gain, basically trading memory (which I have lots of) for
> > >> speed. What is the opinion on a new socket option to let users set
> > >> the size of the receive buffer? I think the size depends on the
> > >> expected data rate and message size, so it seems fair for users to be
> able to
> > influence it.
> > >>
> > >>
> > >>
> > >> Best wishes,
> > >>
> > >>   Jens
> > >>
> > >>
> > >>
> > >> --
> > >>
> > >> Dr. Jens Auer | CGI | Software Engineer
> > >>
> > >> CGI Deutschland Ltd. & Co. KG
> > >> Rheinstraße 95 | 64295 Darmstadt | Germany
> > >>
> > >> T: +49 6151 36860 154
> > >>
> > >> jens.auer at cgi.com
> > >>
> > >> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> > >> unter de.cgi.com/pflichtangaben.
> > >>
> > >>
> > >>
> > >> CONFIDENTIALITY NOTICE: Proprietary/Confidential information
> > >> belonging to CGI Group Inc. and its affiliates may be contained in
> this message.
> > >> If you are not a recipient indicated or intended in this message (or
> > >> responsible for delivery of this message to such person), or you
> > >> think for any reason that this message may have been addressed to you
> > >> in error, you may not use or copy or deliver this message to anyone
> else.
> > >> In such case, you should destroy this message and are asked to notify
> the
> > sender by reply e-mail.
> > >>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> zeromq-dev mailing list
> > >> zeromq-dev at lists.zeromq.org
> > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > >>
> > > _______________________________________________
> > > zeromq-dev mailing list
> > > zeromq-dev at lists.zeromq.org
> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > > _______________________________________________
> > > zeromq-dev mailing list
> > > zeromq-dev at lists.zeromq.org
> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150924/16162be0/attachment.htm>


More information about the zeromq-dev mailing list