[zeromq-dev] Increasing transparently ZMQ performance up to 2400%

Guillermo Lopez Taboada guillermo.lopez.taboada at gmail.com
Fri Feb 28 20:10:06 CET 2014


Torus UFS can run on any machine whereas AFAIK OpenOnload requires a
Solarflare NIC and VMA a Mellanox NIC.

Moreover, UFS takes advantage of RDMA whereas OpenOnload and VMA not. UFS
network support is built on Accelio (accelio.org).

Additionally, UFS achieves latencies (230 nanoseconds intra-server and 1.1
microseconds inter-server) better than what you can see on OpenOnload and
VMA.

Finally, we believe that high-speed networks like Mellanox and Solarflare
will benefit most of your systems so UFS and Mellanox/Solarflare are a
great combination! An example are the results we reported on Mellanox.

Regards,

Guillermo


On Fri, Feb 28, 2014 at 5:41 PM, Steven McCoy <steven.mccoy at miru.hk> wrote:

> The obvious question is what is the relationship between Torus UFS,
> Solarflare OpenOnLoad, and Mellanox VMA?
>
>
> On 28 February 2014 12:37, Guillermo Lopez Taboada <
> guillermo.lopez.taboada at gmail.com> wrote:
>
>>
>> Hi Pieter,
>>
>> Our sockets have intra-server latencies (RTT/2 from process to process)
>> as low as 230 nanoseconds and inter-servers as low as 1.1 microsecond. This
>> makes any socket-based communication-intensive run very very fast. The
>> response time and the scalability can improve significantly so we can
>> alleviate virtually any communication bottleneck.
>>
>> BTW, some people suggested we should include IPC in our tests. We have
>> not included IPC as the performance compared to TCP loopback is quite
>> similar, but, if you do not agree with that and see benefits of using IPC,
>> then congratulations, UFS is for you!
>>
>> Results of a test. For local_lat & remote_lat with 128 bytes we have seen
>>
>> TPC loopback:  average latency: 14.295 [us]
>>
>> IPC: average latency: 12.628 [us]
>>
>> UFS smdev: average latency: 4.570 [us]
>>
>> Regards,
>>
>> Guillermo
>>
>>
>> On Thu, Feb 27, 2014 at 3:21 PM, Pieter Hintjens <ph at imatix.com> wrote:
>>
>>> Guillermo,
>>>
>>> It's a very nice technical demo. Are there other problems such as the
>>> infamous 10 million connections that you also solve?
>>>
>>> -Pieter
>>>
>>> On Thu, Feb 27, 2014 at 2:14 AM, Guillermo Lopez Taboada
>>> <guillermo.lopez.taboada at gmail.com> wrote:
>>> >
>>> > Dear all,
>>> >
>>> > Just let me complete my last message with regards to some questions
>>> that
>>> > came out of the list:
>>> >
>>> > 1) We are accelerating ZeroMQ by using a high performance socket
>>> library in
>>> > user land, UFS, instead of relying on the OS for TCP/IP. UFS is not a
>>> > replacement of ZeroMQ, is an add-on, a plug-in for any socket-based
>>> > application. The way it works is as a preloadable library.
>>> >
>>> > 2) The usage is pretty simple, running a performance test on a server:
>>> >
>>> > # ZeroMQ over TCP loopback:
>>> > $  ./local_lat tcp://127.0.0.1:5555 1 100000  &
>>> > $  ./remote_lat tcp://127.0.0.1:5555 1 100000
>>> > message size: 1 [B]
>>> > roundtrip count: 100000
>>> > average latency: 14.041 [us]
>>> >
>>> >
>>> > # ZeroMQ over UFS (application transparent):
>>> > $ LD_LIBRARY_PATH=$UFS_HOME/lib64 LD_PRELOAD=$UFS_HOME/lib64/libufs.so
>>> > ./local_lat tcp://127.0.0.1:5555 1 100000 &
>>> > $ LD_LIBRARY_PATH=$UFS_HOME/lib64 LD_PRELOAD=$UFS_HOME/lib64/libufs.so
>>> > ./remote_lat tcp://127.0.0.1:5555 1 100000
>>> > message size: 1 [B]
>>> > roundtrip count: 100000
>>> > average latency: 4.830 [us]
>>> >
>>> > 3) Free demos of the software are available by request at
>>> info at torusware.com
>>> >
>>> > Best regards,
>>> >
>>> > Guillermo
>>> >
>>> >
>>> > On Tue, Feb 25, 2014 at 11:06 PM, Guillermo Lopez Taboada
>>> > <guillermo.lopez.taboada at gmail.com> wrote:
>>> >>
>>> >>
>>> >> Dear all,
>>> >>
>>> >> Up to 24x performance increase is the outcome of our latest PoC for a
>>> >> major investment bank.
>>> >>
>>> >> The details are here:
>>> >> http://torusware.com/increase-zeromq-performance-by-up-to-2400/
>>> >>
>>> >> Let us know if you want to put your servers at full speed.
>>> >
>>> >
>>> > --
>>> > Best regards,
>>> >
>>> > Guillermo
>>> > ---
>>> > Guillermo Lopez Taboada
>>> > Torus Software Solutions, CEO
>>> > Email:  guillermo.lopez at torusware.com
>>> > Phone: +34-657-662-998
>>> > WWW: http://www.torusware.com
>>> >
>>> > DISCLAIMER:
>>> > This message is intended exclusively for its addressee and may contain
>>> > information that is CONFIDENTIAL and protected by professional
>>> privilege. If
>>> > you are not the intended recipient you are hereby notified that any
>>> > dissemination, copy or disclosure of this communication is strictly
>>> > prohibited by law. If this message has been received in error, please
>>> > immediately notify us via e-mail and delete it. Thank you.
>>> >
>>> > _______________________________________________
>>> > zeromq-dev mailing list
>>> > zeromq-dev at lists.zeromq.org
>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> >
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Guillermo
>> ---
>> Guillermo Lopez Taboada
>> Torus Software Solutions, CEO
>> Email:  guillermo.lopez at torusware.com
>> Phone: +34-657-662-998
>> WWW: http://www.torusware.com
>>
>>
>> *DISCLAIMER:This message is intended exclusively for its addressee and
>> may contain information that is CONFIDENTIAL and protected by professional
>> privilege. If you are not the intended recipient you are hereby notified
>> that any dissemination, copy or disclosure of this communication is
>> strictly prohibited by law. If this message has been received in error,
>> please immediately notify us via e-mail and delete it. Thank you.*
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>


-- 
Best regards,

Guillermo
---
Guillermo Lopez Taboada
Torus Software Solutions, CEO
Email:  guillermo.lopez at torusware.com
Phone: +34-657-662-998
WWW: http://www.torusware.com


*DISCLAIMER:This message is intended exclusively for its addressee and may
contain information that is CONFIDENTIAL and protected by professional
privilege. If you are not the intended recipient you are hereby notified
that any dissemination, copy or disclosure of this communication is
strictly prohibited by law. If this message has been received in error,
please immediately notify us via e-mail and delete it. Thank you.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140228/01cd8f93/attachment.htm>


More information about the zeromq-dev mailing list