[zeromq-dev] JeroMQ/ZeroMQ transparent acceleration with Fast Sockets

Pieter Hintjens ph at imatix.com
Tue Oct 22 19:26:07 CEST 2013


Hi Guillermo,

Interesting stuff. Is UFS an add-on module on Linux that replaces
standard sockets? It would be interesting if there was a free version
available for people to test.

I see the performance difference for C++ over Infiniband is about 30%.
It would be useful to have figures for throughput, and also comparison
of real TCP (not loopback) on normal sockets vs. UFS.

-Pieter

On Tue, Oct 22, 2013 at 6:43 PM, Guillermo Lopez Taboada
<guillermo.lopez.taboada at gmail.com> wrote:
>
> Dear all,
>
> Torus Software Solutions (http://www.torusware.com) is a company that
> develops transparent replacements of TCP sockets. Currently two
> implementations are in our catalog:
>    - Java Fast Sockets (JFS): a transparent replacement of JVM sockets,
> commercially available.
>    - Universal Fast Sockets (UFS): a system sockets library, currently in
> beta.
>
> Both JFS and UFS accelerate transparently JeroMQ and ZeroMQ over TCP,
> respectively. The peformance benefit according to the benchmarks (see below)
> is around 3 times faster communications both on loopback and InfiniBand.
>
> Both JFS and UFS operate at user-level, are easy to install and work
> immediately with existing ZeroMQ/JeroMQ setups, no recoding nor modification
> of your ZeroMQ/JeroMQ installation is required.
>
> Is there anyone interested in trying out JFS/UFS?
>
>   - JFS is available at http://torusware.com/nevonproducts/jfs/
>   - UFS is available only through our Early Adopters Program. Contact me for
> further details.
>
> Please, feel free to let me know your thoughts about our products and
> approach.
>
> Best regards,
>
> Guillermo
>
>
> ## Machine spec **********************************
>
>
> 2 Dell PowerEdge R620x8 servers with:
>
> * Intel Sandy Bridge E5-2643 at 3.30GHz
>
> * 1600 DDR 32 GB
>
> * Mellanox ConnectX-3 VPI
>
>
> Software:
>
> * Red Hat Linux 6.2 (kernel 2.6.32-358)
>
> * OpenJDK 1.7 JVM
>
> * JeroMQ 3.0.0
>
> * ZeroMQ 3.2.4
>
>
> ## TCP Latency (Using 127.0.0.1) *********************
>
>
> ZeroMQ (C++)
>
>
>    $ ./local_lat tcp://127.0.0.1:1234 <message_size> 1000000 (bind)
>
>    $ ./remote_lat tcp://127.0.0.1:1234 <message_size> 1000000 (connect)
>
>    message_size [b]    10      100     1 k     10 k    100 k
>
>    avg_latency [us]    14.225  15.284  16.007  26.961  46.865
>
>
> ZeroMQ (C++) with UFS
>
>
>    $ ./local_lat tcp://127.0.0.1:1234 <message_size> 1000000 (bind)
>
>    $ ./remote_lat tcp://127.0.0.1:1234 <message_size> 1000000 (connect)
>
>    message_size [b]    10      100     1 k     10 k    100 k
>
>    avg_latency [us]    4.492   4.764   5.064   7.796   17.602
>
>
> JeroMQ
>
>
>    $ java perf.LatencyServer <message_size> 1000000 tcp://127.0.0.1:1234
> (bind)
>
>    $ java perf.LatencyClient 1000000 tcp://127.0.0.1:1234 (connect)
>
>    message_size [b]    10      100     1 k     10 k    100 k
>
>    avg_latency [us]    18.910  19.001  19.956  37.553  85.696
>
>
> JeroMQ with JFS
>
>
>    $ java perf.LatencyServer <message_size> 1000000 tcp://127.0.0.1:1234
> (bind)
>
>    $ java perf.LatencyClient 1000000 tcp://127.0.0.1:1234 (connect)
>
>    message_size [b]   10      100     1 k     10 k    100 k
>
>    avg_latency [us]   3.396   3.304   4.202   7.614   31.765
>
>
>
>
> ## InfiniBand Latency *************************************
>
>
> ZeroMQ (c++)
>
>
>    $ ./local_lat tcp://<ib_ip>:1234 <message_size> 1000000 (bind)
>
>    $ ./remote_lat tcp://<ib_ip>:1234 <message_size> 1000000 (connect)
>
>    message_size [b]    10      100     1 k     10 k    100 k
>
>    avg_latency [us]    17.401  17.905  19.321  31.185  76.595
>
>
> ZeroMQ (c++) with UFS
>
>
>    $ ./local_lat tcp://<ib_ip>:1234 <message_size> 1000000 (bind)
>
>    $ ./remote_lat tcp://<ib_ip>:1234 <message_size> 1000000 (connect)
>
>    message_size [b]    10      100     1 k     10 k    100 k
>
>    avg_latency [us]    11.918  11.678  13.978  23.637  72.325
>
>
> JeroMQ
>
>
>    $ java perf.LatencyServer <message_size> 1000000 tcp://<ib_ip>:1234
> (bind)
>
>    $ java perf.LatencyClient 1000000 tcp://<ib_ip>:1234 (connect)
>
>    message_size [b]    10      100     1 k     10 k    100 k
>
>    avg_latency [us]    22.415  24.222  25.270  40.103  114.448
>
>
> JeroMQ with JFS
>
>
>    $ java perf.LatencyServer <message_size> 1000000 tcp://<ib_ip>:1234
> (bind)
>
>    $ java perf.LatencyClient 1000000 tcp://<ib_ip>:1234 (connect)
>
>    message_size [b]   10      100     1 k     10 k    100 k
>
>    avg_latency [us]   11.745  12.571  13.496  29.396  169,022
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
> Guillermo
> ---
> Torus Software Solutions, CEO
> Email:  guillermo.lopez at torusware.com
> WWW: http://www.torusware.com
>
> DISCLAIMER:
> This message is intended exclusively for its addressee and may contain
> information that is CONFIDENTIAL and protected by professional privilege. If
> you are not the intended recipient you are hereby notified that any
> dissemination, copy or disclosure of this communication is strictly
> prohibited by law. If this message has been received in error, please
> immediately notify us via e-mail and delete it. Thank you.
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



-- 
-
Pieter Hintjens
CEO of iMatix.com
Founder of ZeroMQ community
blog: http://hintjens.com



More information about the zeromq-dev mailing list