[zeromq-dev] Performance results on 100Gbps direct link

Francesco francesco.montorsi at gmail.com
Mon Aug 12 11:04:43 CEST 2019


Hi Benjamin,

Il giorno dom 11 ago 2019 alle ore 13:05 Benjamin Henrion
<zoobab at gmail.com> ha scritto:
> Do you have a test which can show first that you can saturate the 100gbps link?
>
> Like an iperf or a simple wget test?

I don't have a graph just like the one I created with ZMQ benchmark
utilities but using a custom DPDK app creating packets of size
256-512B I've been able to saturate the 100Gbps link using the 16 CPU
cores of the first CPU, with hyperthreading off (the server has 2
CPUs/NUMA nodes, each with 16 physical cores that become 32 with HT
for a total of 64).
This is not that far from the official Mellanox reports using DPDK:
  https://fast.dpdk.org/doc/perf/DPDK_19_05_Mellanox_NIC_performance_report.pdf
  (see chapter 5)
where they declare they can the line rate of a 100Gbps link using 12
cores (well, they're using Xeon Platinum CPUs while my server had Gold
CPUs and moreover the DPDK application they used, l3fwd, is very
simple indeed).
And yes: the line rate for 64B packets at 100Gbps is an astonishingly
148 million packets per seconds.

Of course I don't think we can ever reach that performances with using
regular Linux kernel TCP stack (btw that DPDK example sends frames
over IPv4 but I don't know if over IPv4 it uses UDP, TCP or something
else).
However currently ZMQ does about 1Mpps @ 64B which is pretty far from
that 148Mpps... I wonder if that could be improved :)

As you point out it would be interesting to run a test using iperf or
wget that indeed use the Linux kernel TCP stack as well... when I will
have access to that setup I can try.

Francesco



More information about the zeromq-dev mailing list