[zeromq-dev] ZeroMQ for data transfer (large messages)

Pieter Hintjens ph at imatix.com
Fri Aug 29 18:44:44 CEST 2014


When you send large messages the middleware becomes more and more
irrelevant for performance, as the cost per message drops and the
network becomes the bottleneck. Thus the fastest protocol for file
transfer is probably FTP.

What's interesting is how easily you can build your protocol on top of
the messaging. I'd suggest you read the ZeroMQ Guide with particular
regard to FILEMQ.

-Pieter



On Fri, Aug 29, 2014 at 4:00 PM, Aram Santogidis
<aram.santogidis at cern.ch> wrote:
> Hi all,
>
>
>
> I’m experimenting with ZeroMQ and NanoMSG and I would like to ask your input
> on the following topic.
>
>
>
> I encountered various performance evaluation data for 0MQ and other
> message-oriented middleware, and I noticed that the throughput sections
> focus quite frequently in the range of at most tens of KiloBytes of message
> sizes. Why is that? This range of message sizes is typical for applications
> based on message-oriented middleware? Maybe it is assumed that for large
> volume data transfers (in the order of hundreds  of megabytes), other
> (non-MOM) mechanisms should be employed?
>
>
>
> It happens to be that I am interested to move data (as single messages) in
> the order of tens (or hundreds) of megabytes. One question is if ZeroMQ (and
> NanoMSG) was meant to be used for this kind of workloads. I know that it is
> possible (I’ve also seen the performance reports published at the site for
> up to 512 MB message sizes) but is it natural? By “it was meant to be used
> for this kind of workloads” I mean weather performance optimization(besides
> zero-copy) are focused for this kind of range of messages [10MB-1GB] as
> well.
>
>
>
> I would appreciate your opinion on this topic.
>
>
>
> Thank you,
>
> Aram
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



More information about the zeromq-dev mailing list