[zeromq-dev] Anybody with updated performance results http://zeromq.org/area:results ?

Francesco francesco.montorsi at gmail.com
Wed Aug 7 23:35:42 CEST 2019 

Hi Brett,

>- The page shows PUB/SUB on inproc://, how does PUB/SUB perform on 10  GbE
tcp://?

Unfortunately there is no benchmark utility for PUB/SUB which is what I use
all the times - so I would be curious to try... would ZeroMQ developers
accept a merge request to allow local_thr/remote_thr to take from command
line the socket pair to use for testing (instead of hardcoding the
PUSH/PULL pair)?

> - How are dropped messages accounted in the PUB/SUB tests?

If you are referring the PUB/SUB proxy throughput graph, consider it's
generated by this util
  https://github.com/zeromq/libzmq/blob/master/perf/proxy_thr.cpp
which uses ZMQ_XPUB_NODROP=1 so there are no drops at all :)

> And, one thing I don't understand (maybe just a curiosity): PUSH/PULL
shows somewhat better PPS throughput with tcp:// than with inproc:// for
> messages between 100-1000 Bytes.  Naively, I'd have thought shared memory
would beat network for message throughput for any given message size.

Yeah, good point... probably the graph needs some "zoom-in" to make that
more evident but anyway I would propose the following explanation: TCP
transport has much more buffer than INPROC transport (because beside the
HWM buffers owned by ZMQ it also has TCP kernel buffers to take in count...
on those boxes with plenty of RAM the kernel socket buffers can be "big"
like 16MB)... perhaps that explains why with TCP the NIC always has its TX
queues filled and finally achieves a higher PPS ? Not sure...

Francesco




Il giorno mer 7 ago 2019 alle ore 17:32 Brett Viren <bv at bnl.gov> ha scritto:

> This is very useful information.  Thank you for sharing it.
>
> I have some 10 GbE hardware on order and hope to reproduce this myself
> and answer these questions but I'm curious to know:
>
> - The page shows PUB/SUB on inproc://, how does PUB/SUB perform on 10
>   GbE tcp://?
>
> - How are dropped messages accounted in the PUB/SUB tests?
>
>
> And, one thing I don't understand (maybe just a curiosity): PUSH/PULL
> shows somewhat better PPS throughput with tcp:// than with inproc:// for
> messages between 100-1000 Bytes.  Naively, I'd have thought shared
> memory would beat network for message throughput for any given message
> size.
>
> Thanks again,
> -Brett.
>
> Francesco <francesco.montorsi at gmail.com> writes:
>
> > Hi Luca, Hi all,
> >
> > I generated the results graph and put all of them here:
> >
> >  http://zeromq.org/results:10gbe-tests-v432
> >
> > I would say the results are ok but perhaps there's room for improvements.
> > For example: the local_thr/remote_thr benchmarks show that ZeroMQ is
> able to fill the
> > 10Gbps link only using message sizes of about 10kB.
> > The CPUs of the test spiked at about 3.5 Mpps @ 16B message-size  which
> is a bit far
> > from the theoretical max of Ethernet that for 84B frames (on the wire)
> is 14.8Mpps
> > (see https://kb.juniper.net/InfoCenter/index?page=content&id=KB14737).
> >
> > I wonder how ZeroMQ message batching mechanism works for small messages
> (<1kB)
> > on TCP... anybody can shed some light on this? Thanks!
> >
> > Francesco
> >
> > PS: any project to use something like F-stack (http://www.f-stack.org/)
> on top of DPDK
> > as backend for ZeroMQ :) ?
> >
> > Il giorno dom 4 ago 2019 alle ore 20:41 Luca Boccassi <
> luca.boccassi at gmail.com> ha
> > scritto:
> >
> >  Looks great, thank you!
> >
> >  On Sun, 4 Aug 2019, 18:28 Francesco, <francesco.montorsi at gmail.com>
> wrote:
> >
> >  Hi,
> >
> >   > There's nothing that I know of for that purpose
> >
> >  I wrote a 70lines bash script to automate the collection of performance
> results
> >  using "{local/remote/inproc/proxy}_{thr/lat}" ZMQ performance utils...
> >  I created a PR for that: https://github.com/zeromq/libzmq/pull/3607
> >
> >  Let me know if that works for you.
> >
> >  As soon as I have the HW available I will use them to generate the new
> >  graphs...
> >
> >  Thanks
> >
> >  Francesco
> >
> >  Il giorno sab 3 ago 2019 alle ore 11:39 Luca Boccassi
> >  <luca.boccassi at gmail.com> ha scritto:
> >
> >  There's nothing that I know of for that purpose
> >
> >  On Sat, 3 Aug 2019, 10:24 Francesco, <francesco.montorsi at gmail.com>
> >  wrote:
> >
> >  Hi Luca,
> >  I don't have a wikidot account... however I have a basic question before
> >  getting there:
> >     local_thr / remote_thr
> >  utilities are just producing a text output... is there any script to:
> >  1) run them automatically to generate all points of the
> >  per-message-size graphs (http://zeromq.org/results:10gbe-tests-v031)
> >  ?
> >  2) produce the actual graph from the collected text outputs ?
> >
> >  Thanks,
> >  Francesco
> >
> >  Il giorno sab 3 ago 2019 alle ore 00:40 Luca Boccassi
> >  <luca.boccassi at gmail.com> ha scritto:
> >
> >  Yes please!
> >
> >  Do you have an account on wikidot to edit the page?
> >
> >  On Fri, 2 Aug 2019, 21:54 Francesco,
> >  <francesco.montorsi at gmail.com> wrote:
> >
> >  Hi all,
> >  I noticed that all performance results reported at this page:
> >    http://zeromq.org/area:results
> >  seem a bit outdated (most updated version looks like
> >  ØMQ/2.0.6 !)... has anybody updated results?
> >  Alternatively I may be able to generate measurements on
> >  latest libzmq on 10G NICs... would you be interested in putting
> >  on that page updated results?
> >
> >  Thanks,
> >  Francesco
> >
> >  _______________________________________________
> >  zeromq-dev mailing list
> >  zeromq-dev at lists.zeromq.org
> >  https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >  _______________________________________________
> >  zeromq-dev mailing list
> >  zeromq-dev at lists.zeromq.org
> >  https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >  _______________________________________________
> >  zeromq-dev mailing list
> >  zeromq-dev at lists.zeromq.org
> >  https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >  _______________________________________________
> >  zeromq-dev mailing list
> >  zeromq-dev at lists.zeromq.org
> >  https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >  _______________________________________________
> >  zeromq-dev mailing list
> >  zeromq-dev at lists.zeromq.org
> >  https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >  _______________________________________________
> >  zeromq-dev mailing list
> >  zeromq-dev at lists.zeromq.org
> >  https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20190807/cdd8a32d/attachment.htm>

More information about the zeromq-dev mailing list