[zeromq-dev] Reduce latency on Pub-Sub Broker

[Robert Evans]

Hi Pieter,

I ran your suggested C latency tests and the results seemed to be okay.

local_lat and remote_lat with 10 bytes message with 100 000 roundtrips
showed a latency of 43 microseconds, inproc_lat 16 microseconds. I think
the problem is my broker. When I used a direct connection between publisher
with HWM=500 and subscriber without a broker the latency was about 0,3
milliseconds. Once I added the broker again, the latency increased to ~15
milliseconds.

My CPU has 4 cores and the network is capable of 100 mbit/s. I'm currently
running everything (pub, sub and broker) on the same machine. Maybe this
leads to some problems with the CPU or the network. I will try moving the
broker to an own machine and see what will happen. I once had an experience
with AleriMQ, when the latency was significantly lower when the broker ran
on a seperate machine - perhaps it could be the same here.

Regards,
Robert

2012/5/29 Pieter Hintjens <ph at imatix.com>

> Robert,
>
> First, try some tests in plain C (using the low level API or CZMQ), to
> make sure that it's not the C# binding adding latency.
>
> There are command-line test programs you can use (perf/local_lat,
> perf/remote_lat).
>
> Secondly, if the latency increases when you send messages faster,
> there is something bogus happening. 0MQ will normally batch small
> messages which reduces latency if you send larger volumes. The main
> reason for setting a HWM is if your subscriber or network is too slow,
> and you need to guard against memory exhaustion.
>
> It sounds like you have a CPU contention issue. How many cores on your
> box? What kind of network?
>
> -Pieter
>
>
>
> On Tue, May 29, 2012 at 10:26 AM, Robert Evans
> <zwergzebra at googlemail.com> wrote:
> > Hello everyone,
> >
> > I am currently developing a Publish-Subscribe broker using the C# binding
> > clrzmq v. 2.2.3. My operating system is Windows 7 Prof. and my system is
> a
> > Intel i7 with 16 GByte RAM.
> >
> > The broker works without any problems, I have added one publisher and one
> > subscriber and get message rates about 250 000 - 150 000 msg/second when
> > sending 10 byte messages on localhost. The broker simply forwards the
> > messages from the publisher to the subscriber, no big magic there.
> >
> > My problem is the increasing latency when sending many messages at short
> > time. When I only send one message and sleep for one millisecond the
> latency
> > is < 1ms, but the message rate drops to 9000 msg/s. When I'm sending
> > messages straight without sleeping at ~ 200 000 msg/s, the latency
> increases
> > with the number of messages sent. I set the HWM on the broker for the pub
> > and sub socket to 500, which seems to narrow the latency to 10 - 20 ms.
> This
> > is okay, but a value between 1 and 3 ms would be even better.
> >
> > Is there a way to reduce it even more? I don't really need the queuing of
> > messages with the HWM; I would rather like to drop messages instantly
> > instead of queuing as low latency is very important in my project (but I
> > still need a message rate about 100 000 msg/s). As to say shortly: Better
> > drop some messages for lower latency, instead of buffering them.
> >
> > It would be great if someone has an idea or already has some experience
> with
> > tweaking the latency. Thank you!
> >
> > Regards,
> > Robert
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20120529/c234e400/attachment.htm>