[zeromq-dev] Mb vs. MB in tests & on the Wiki

[Holger Hoffstätte]

Martin Sustrik wrote:
>> I was just having some more fun with the zmq examples when I noticed that
>> all the tests and also the Wiki use [Mb] for throughput. Not to be overly
>> pedantic but unless I'm mistaken that should probably be [MB] since it's
>> megabytes, not -bits. My poor little box is slow but it's certainly
>> better
>> than ~60 MB/sec over loopback. ;)
> 
> It's Mb, not MB. The point is to make it comparable with bandwidth of
> your network interfaces. For example, if you are running on 1Gb
> Ethernet, it's useful to know that you are able to exhaust say 534 Mb/s
> which translates in straightforward way to 53.4%. Reporting it as 66.7
> MB/s is a little bit less useful.

I understand the need to compare to NICs and possible bandwidth etc.
though in my head I still have to think in kilo/megabytes etc.. but OK,
just wanted to make sure there's no typo or confusion since most people
get the units wrong. Thanks for confirming.

> As for performance of your loopback interface, it depends on the message
> size you are testing. Say you are testing 1 byte long messages. Thus
> 60MB/s would give 60,000,000 messages a second which is above all limits
> for today's middleware.

Sure, I understand CPU overhead, packet sizes etc. - I was testing with
500/1k/4k/larger messages and extrapolating from the number of messages.
Guess I need to run those tests again..

[5 mins. later..]

..sigh! You were right of course. I keep getting 60-70 MB/sec with the
local_thr test over localhost:lo and messages from ~1k - 8k. With 32k and
above it levels out at ~100MB/sec and sometimes fails, I assume with OOMs
(zmq::raw_message_init: no msg_->content).

I know loopback is not realistic since it is mostly optimized away by the
kernel and everything becomes an exercise in context switching and memory
copying, but it's the best I can do considering my Intel e1000 Gb NIC is
sadly stuck in plain PCI and won't push more than 75MB/sec no matter how
hard I threaten it.

FWIW lmbench says this rusty tin pushes ~1.1GB (not bit) between processes
with AF_UNIX but of course that is without zmq overhead. Still, shouldn't
throughput over loopback be higher than ~60-70 MB even with only one core?

Then again, latency is much more interesting to me anyway. I'll run some
tests on a dual-core machine with the -rt kernel later and can report back
if it yields anthing useful.

Btw I'm doing this just for fun to see how much I can squeeze out of my
old gear, so it's not really critical. :-)

thanks & regards
Holger