[zeromq-dev] Mb vs. MB in tests & on the Wiki

[Martin Sustrik]

Holger,

> ..sigh! You were right of course. I keep getting 60-70 MB/sec with the
> local_thr test over localhost:lo and messages from ~1k - 8k. With 32k and
> above it levels out at ~100MB/sec

I assume you've meant Mb (this B/b thing is really confusing) which 
equals to some 7500 messages a second. Even with 60MB is would be 60,000 
msgs/sec which is really unacceptable.

On what kind of hardware are you running?

I've tried to run on loopback on a single-core P4 box and this is what 
I've got:

threads: 1
message size: 1024 [B]
message count: 1000000
Your average throughput is 196822 [msg/s]
Your average throughput is 1612 [Mb/s]

An idea: What message count are you using? Small amount of messages in 
the test (<100000) may cause throughput figure to change unpredictably 
from run to run...

> and sometimes fails, I assume with OOMs
> (zmq::raw_message_init: no msg_->content).

We haven't seen this kind of behaviour. Looks like some nasty bug. Would 
you mind running debug version (export CXXFLAGS="-g -O0") under gdb and 
sending us backtraces? Optionally, you can report the bug at 
jira.fastmq.org.

> I know loopback is not realistic since it is mostly optimized away by the
> kernel and everything becomes an exercise in context switching and memory
> copying, but it's the best I can do considering my Intel e1000 Gb NIC is
> sadly stuck in plain PCI and won't push more than 75MB/sec no matter how
> hard I threaten it.

Loopback messaging is one of the scenarios we should take into account 
so it's good you are testing it. From our limited experience loopback 
interface tends to be rather unstable from performance point of view, 
i.e. results differ wildly among the runs.

> FWIW lmbench says this rusty tin pushes ~1.1GB (not bit) between processes
> with AF_UNIX but of course that is without zmq overhead. Still, shouldn't
> throughput over loopback be higher than ~60-70 MB even with only one core?

Yup. See the figures above.

> Then again, latency is much more interesting to me anyway. I'll run some
> tests on a dual-core machine with the -rt kernel later and can report back
> if it yields anthing useful.

We are doing tests on RT kernel right now, so you joining the effort 
would be helpful.

In overall it looks like you won't see better latencies for RT kernel. 
Presumably, they'll be a little bit hgiher, however, the jitter should 
be  much lower. To test this you need a specific test case that measures 
latency for *each* message + a statistical tool to compute metrics from 
the mass of output data. We can share that with you in case you are 
interested.

Martin