[zeromq-dev] Measuring Performance Publish & Subscribe

Aaron Sokoloski asokoloski at gmail.com
Mon Jul 18 14:02:49 CEST 2016


1000 is the default High Water Mark, so it sounds suspiciously like some of
your messages are getting dropped.  Does your latency measurement make any
assumptions about the messages all being present which might cause it to
show nonsense if some messages are dropped?

Perhaps try with PUSH / PULL instead of PUB / SUB and see if that changes
the behaviour.

I'm not sure I understand your question, sorry...  PUB / SUB does not
block, but if the subscriber cannot process messages fast enough then some
will get dropped.  There is a buffer within zeromq, but there is also the
OS TCP buffer, so it can be difficult to predict how far behind the
subscriber will get before dropping messages.

Cheers,
Aaron

On 18 July 2016 at 06:51, Sta318 at web.de <sta318 at web.de> wrote:

> Hi Aaron,
>
> first thanks for your answer. Yes, that are 11 seconds. I am logging the
> time for Protobuf, too. There is noticeable measuremt. It has quite
> constant values.
> The subscriber received 1000 messages. After the first message has a
> duration of 11 seconds, every following message has a duration of 11
> seconds. But this is not logical, because the whole measuremt doesn't take
> 500 * 11 seconds.
>
> Towards this behaviour, i have a general understanding question. As i
> understand the publish and subscribe pattern is asynchronous. The send
> method is nonblocking. When there is no speed limitation on sending then
> the messagequeue gets filled immediately. Is this correct? When it is
> correct the time that is measured has to look incremented.
> I don't understand why i get a constant value for measuring without
> limitation for sending frequency.
>
> Sry, typing from smartphone. You can delete zeromq-dev Digest, Vol 4,
> Issue 10
>
> Regards
> Sebastian
>
> Am 18. Juli 2016 12:00:01 MESZ, schrieb
> zeromq-dev-request at lists.zeromq.org:
>>
>> Send zeromq-dev mailing list submissions to
>>  zeromq-dev at lists.zeromq.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> or, via email, send a message with subject or body 'help' to
>>  zeromq-dev-request at lists.zeromq.org
>>
>> You can reach the person managing the list at
>>  zeromq-dev-owner at lists.zeromq.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of zeromq-dev digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Re: Measuring Performance Publish & Subscribe (Aaron Sokoloski)
>>
>>
>> ------------------------------
>>
>>
>> Message: 1
>> Date: Mon, 18 Jul 2016 04:58:58 -0500
>> From: Aaron Sokoloski <asokoloski at gmail.com>
>> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
>> Subject: Re: [zeromq-dev] Measuring Performance Publish & Subscribe
>> Message-ID:
>>  <CAAajcW6xxbrYxtoykCx3m6kiHDKZf-dneBpsY6L5UoPfi-=ATw at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>>
>> Hi Sebastian,
>>
>> Here are the questions I would ask if I were trying to figure this out:
>>
>> Those numbers look to be about 11 seconds.  Are they correct?  When you run
>> the program and watch the log, do you see it pause for about that much time?
>> Does it have anything to do with deserialising the large protobuf?  If you
>> skip that step, or avoid protobuf entirely, does the problem disappear?
>> Are any messages getting dropped?  PUB/SUB can drop messages if the
>> receiver can't process them fast enough.  If a message got dropped, would
>> that explain the latency?  Are all the messages slow after that first slow
>> one, or does it go back to being quick?
>>
>> Cheers,
>> Aaron
>>
>>
>> On 17 July 2016 at 03:10, <sta318 at web.de> wrote:
>>
>>  Hi Team,
>>>
>>>
>>>
>>>  i am measuring the latency of the publish & suscribe pattern using
>>>  protobuf. I am using zeromq 4.1.3 on an windows 7 embedded. The publisher &
>>>  subscriber are written in C++.
>>>
>>>
>>>
>>>  The publisher and the subscriber are on the same system. I wrote two
>>>  protos which both use. For measuring the latency i am using the Windows
>>>  High Precision Event Timer.
>>>
>>>  In one protobuf object the data is stored. In the other protobuf object
>>>  the systemclocks from the publisher are stored in. On the publisher site
>>>  the system cycles after the start are wrote in the protobuf object. The
>>>  objects are sent in a multipart message. When the subscriber gets the
>>>  messages, he calculates the difference and gets the latency in microseconds.
>>>
>>>
>>>
>>>  When the payload is maller than 300.12 kB i get logic results. But when
>>>  the payload is 300.12 kB or bigger i get this huge results.
>>>
>>>
>>>
>>>  Payload (kB)  AverageLatency (us)
>>>
>>>  300.12            7.8928e+06
>>>
>>>  600.12           5.6801e+06
>>>
>>>  1200.1           5.3435e+06
>>>
>>>  3000.1           5.2595e+06
>>>
>>>
>>>
>>>  When i look directly into the logs i see this behaviour:
>>>
>>>  For a payload of 300.12:
>>>
>>>  MessageNo. Payload (kB)   Latency (us)
>>>
>>>  233                         300.125            3961
>>>
>>>  234                         300.125   11131668
>>>
>>>
>>>
>>>  The same for a Payload of 600.12
>>>
>>>  MessageNo. Payload (kB)   Latency (us)
>>>
>>>  489                       0.600125          8799
>>>
>>>  490                       0.600125 11083799
>>>
>>>
>>>
>>>  The same for a Payload of 1200.13
>>>
>>>  MessageNo. Payload (kB)   Latency (us)
>>>
>>>  548                         1.20013        17555
>>>
>>>  549                         1.20013 11082562
>>>
>>>
>>>
>>>  The same for a Payload of 1200.13
>>>
>>>  MessageNo. Payload (kB)   Latency (us)
>>>
>>>  55                           3.00012       45364
>>>
>>>  56                           3.00012 11164985
>>>
>>>
>>>
>>>  I don’t understand the explosion of the latency. Is there a timeout or
>>>  what happened?
>>>
>>>  What makes ZeroMQ in the background?
>>>
>>>
>>>
>>>  Regards
>>>
>>>  Sebastian
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------
>>>
>>>  zeromq-dev mailing list
>>>  zeromq-dev at lists.zeromq.org
>>>  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160718/82028bc5/attachment-0001.html>
>>
>> ------------------------------
>>
>>
>> Subject: Digest Footer
>>
>> ------------------------------
>>
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> ------------------------------
>>
>>
>> End of zeromq-dev Digest, Vol 4, Issue 10
>> *****************************************
>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160718/afc03aca/attachment.html>


More information about the zeromq-dev mailing list