[zeromq-dev] JeroMQ/ZeroMQ transparent acceleration with Fast Sockets

Guillermo Lopez Taboada guillermo.lopez.taboada at gmail.com
Tue Oct 22 20:11:33 CEST 2013


Hi Pieter!

UFS is a preloadable library that captures standard sockets and uses its
custom protocol in case the communicating peer also has UFS, otherwise it
falls back to TCP sockets.

Right now we are still working on UFS but there is a free version to test,
available on request for anyone interested in playing with UFS.

We will update the figures with throughput results, in this particular case
replacing TCP is not so important for small messages so the benefit is
lower in these cases.

Finally, our libraries replace sockets on loopback and RDMA networks. For
accelerating sockets in other environments I would recommend first buying a
low-latency/RDMA network, as the network card itself would be a major
bottleneck :D

Best regards,

Guillermo



On Tue, Oct 22, 2013 at 7:26 PM, Pieter Hintjens <ph at imatix.com> wrote:

> Hi Guillermo,
>
> Interesting stuff. Is UFS an add-on module on Linux that replaces
> standard sockets? It would be interesting if there was a free version
> available for people to test.
>
> I see the performance difference for C++ over Infiniband is about 30%.
> It would be useful to have figures for throughput, and also comparison
> of real TCP (not loopback) on normal sockets vs. UFS.
>
> -Pieter
>
> On Tue, Oct 22, 2013 at 6:43 PM, Guillermo Lopez Taboada
> <guillermo.lopez.taboada at gmail.com> wrote:
> >
> > Dear all,
> >
> > Torus Software Solutions (http://www.torusware.com) is a company that
> > develops transparent replacements of TCP sockets. Currently two
> > implementations are in our catalog:
> >    - Java Fast Sockets (JFS): a transparent replacement of JVM sockets,
> > commercially available.
> >    - Universal Fast Sockets (UFS): a system sockets library, currently in
> > beta.
> >
> > Both JFS and UFS accelerate transparently JeroMQ and ZeroMQ over TCP,
> > respectively. The peformance benefit according to the benchmarks (see
> below)
> > is around 3 times faster communications both on loopback and InfiniBand.
> >
> > Both JFS and UFS operate at user-level, are easy to install and work
> > immediately with existing ZeroMQ/JeroMQ setups, no recoding nor
> modification
> > of your ZeroMQ/JeroMQ installation is required.
> >
> > Is there anyone interested in trying out JFS/UFS?
> >
> >   - JFS is available at http://torusware.com/nevonproducts/jfs/
> >   - UFS is available only through our Early Adopters Program. Contact me
> for
> > further details.
> >
> > Please, feel free to let me know your thoughts about our products and
> > approach.
> >
> > Best regards,
> >
> > Guillermo
> >
> >
> > ## Machine spec **********************************
> >
> >
> > 2 Dell PowerEdge R620x8 servers with:
> >
> > * Intel Sandy Bridge E5-2643 at 3.30GHz
> >
> > * 1600 DDR 32 GB
> >
> > * Mellanox ConnectX-3 VPI
> >
> >
> > Software:
> >
> > * Red Hat Linux 6.2 (kernel 2.6.32-358)
> >
> > * OpenJDK 1.7 JVM
> >
> > * JeroMQ 3.0.0
> >
> > * ZeroMQ 3.2.4
> >
> >
> > ## TCP Latency (Using 127.0.0.1) *********************
> >
> >
> > ZeroMQ (C++)
> >
> >
> >    $ ./local_lat tcp://127.0.0.1:1234 <message_size> 1000000 (bind)
> >
> >    $ ./remote_lat tcp://127.0.0.1:1234 <message_size> 1000000 (connect)
> >
> >    message_size [b]    10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]    14.225  15.284  16.007  26.961  46.865
> >
> >
> > ZeroMQ (C++) with UFS
> >
> >
> >    $ ./local_lat tcp://127.0.0.1:1234 <message_size> 1000000 (bind)
> >
> >    $ ./remote_lat tcp://127.0.0.1:1234 <message_size> 1000000 (connect)
> >
> >    message_size [b]    10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]    4.492   4.764   5.064   7.796   17.602
> >
> >
> > JeroMQ
> >
> >
> >    $ java perf.LatencyServer <message_size> 1000000 tcp://127.0.0.1:1234
> > (bind)
> >
> >    $ java perf.LatencyClient 1000000 tcp://127.0.0.1:1234 (connect)
> >
> >    message_size [b]    10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]    18.910  19.001  19.956  37.553  85.696
> >
> >
> > JeroMQ with JFS
> >
> >
> >    $ java perf.LatencyServer <message_size> 1000000 tcp://127.0.0.1:1234
> > (bind)
> >
> >    $ java perf.LatencyClient 1000000 tcp://127.0.0.1:1234 (connect)
> >
> >    message_size [b]   10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]   3.396   3.304   4.202   7.614   31.765
> >
> >
> >
> >
> > ## InfiniBand Latency *************************************
> >
> >
> > ZeroMQ (c++)
> >
> >
> >    $ ./local_lat tcp://<ib_ip>:1234 <message_size> 1000000 (bind)
> >
> >    $ ./remote_lat tcp://<ib_ip>:1234 <message_size> 1000000 (connect)
> >
> >    message_size [b]    10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]    17.401  17.905  19.321  31.185  76.595
> >
> >
> > ZeroMQ (c++) with UFS
> >
> >
> >    $ ./local_lat tcp://<ib_ip>:1234 <message_size> 1000000 (bind)
> >
> >    $ ./remote_lat tcp://<ib_ip>:1234 <message_size> 1000000 (connect)
> >
> >    message_size [b]    10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]    11.918  11.678  13.978  23.637  72.325
> >
> >
> > JeroMQ
> >
> >
> >    $ java perf.LatencyServer <message_size> 1000000 tcp://<ib_ip>:1234
> > (bind)
> >
> >    $ java perf.LatencyClient 1000000 tcp://<ib_ip>:1234 (connect)
> >
> >    message_size [b]    10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]    22.415  24.222  25.270  40.103  114.448
> >
> >
> > JeroMQ with JFS
> >
> >
> >    $ java perf.LatencyServer <message_size> 1000000 tcp://<ib_ip>:1234
> > (bind)
> >
> >    $ java perf.LatencyClient 1000000 tcp://<ib_ip>:1234 (connect)
> >
> >    message_size [b]   10      100     1 k     10 k    100 k
> >
> >    avg_latency [us]   11.745  12.571  13.496  29.396  169,022
> >
> >
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> > Guillermo
> > ---
> > Torus Software Solutions, CEO
> > Email:  guillermo.lopez at torusware.com
> > WWW: http://www.torusware.com
> >
> > DISCLAIMER:
> > This message is intended exclusively for its addressee and may contain
> > information that is CONFIDENTIAL and protected by professional
> privilege. If
> > you are not the intended recipient you are hereby notified that any
> > dissemination, copy or disclosure of this communication is strictly
> > prohibited by law. If this message has been received in error, please
> > immediately notify us via e-mail and delete it. Thank you.
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
>
>
>
> --
> -
> Pieter Hintjens
> CEO of iMatix.com
> Founder of ZeroMQ community
> blog: http://hintjens.com
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



-- 
Best regards,

Guillermo
---
Torus Software Solutions, CEO
Email:  guillermo.lopez at torusware.com
Phone: +34-657-662-998
WWW: http://www.torusware.com

*DISCLAIMER:
This message is intended exclusively for its addressee and may contain
information that is CONFIDENTIAL and protected by professional privilege.
If you are not the intended recipient you are hereby notified that any
dissemination, copy or disclosure of this communication is strictly
prohibited by law. If this message has been received in error, please
immediately notify us via e-mail and delete it. Thank you.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131022/e07ee03b/attachment.htm>


More information about the zeromq-dev mailing list