[zeromq-dev] sending 20MB messages using zeromq

Pieter Hintjens ph at imatix.com
Mon May 12 14:48:30 CEST 2014


Performance comes from many things. There's no single answer. At the
least you have to understand the different patterns and how they work.
Please read the Guide in detail and work through the examples, that
will provide a large part of the answer.

On Mon, May 12, 2014 at 2:31 PM, Dileep Mandapam
<dileep.mandapam at techmojo.in> wrote:
> Thanks Pieter,
>
> For single producer and single consumer case , Does PUB-SUB messaging
> pattern gives good performance ?.
>
>
>
>
>
>
> On Mon, May 12, 2014 at 5:48 PM, Pieter Hintjens <ph at imatix.com> wrote:
>>
>> Your performance is almost certainly limited by network bandwidth
>> here. Any other application using the network will impact your
>> throughput. If you want to get better performance, consider a
>> dedicated network, or a faster network.
>>
>> As you're doing pub-sub, if you are sending the same message to many
>> subscribers, consider using a multicast transport like pgm:// or
>> norm://
>>
>> -Pieter
>>
>> On Mon, May 12, 2014 at 1:48 PM, Dileep Mandapam
>> <dileep.mandapam at techmojo.in> wrote:
>> > Hi,
>> >
>> > Use-case :-  I have 'N' request processor threads which sends responses
>> > to 1
>> > node.js server using ZEROMQ PUB/SUB .To improve performance, I have
>> > offloaded publishing task to one dedicated thread ,which keeps on
>> > sending
>> > responses .
>> >
>> > My question is ,Is there any to improve throughput ?
>> >
>> > sample test results :-
>> >
>> > 12 May 2014 13:26:11,199 1239673 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 62 milli seconds for  100 payloads
>> > 12 May 2014 13:26:11,460 1239934 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 44 milli seconds for  100 payloads
>> > 12 May 2014 13:26:11,782 1240256 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 47 milli seconds for  100 payloads
>> > 12 May 2014 13:26:12,132 1240606 [zmqPub--2-thread-1] INFO LOGGER  -
>> > sendResponses took 59 milli seconds for  100 payloads
>> > 12 May 2014 13:26:12,407 1240881 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 45 milli seconds for  100 payloads
>> > 12 May 2014 13:26:12,744 1241218 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 53 milli seconds for  100 payloads
>> > 12 May 2014 13:26:12,789 1241263 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 7 milli seconds for  10 payloads
>> > 12 May 2014 13:26:13,627 1242101 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 207 milli seconds for  100 payloads
>> > 12 May 2014 13:26:14,336 1242810 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 117 milli seconds for  100 payloads
>> > 12 May 2014 13:26:15,073 1243547 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 310 milli seconds for  100 payloads
>> > 12 May 2014 13:26:16,026 1244500 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 225 milli seconds for  100 payloads
>> > 12 May 2014 13:26:16,359 1244833 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 60 milli seconds for  100 payloads
>> > 12 May 2014 13:26:16,661 1245135 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 54 milli seconds for  100 payloads
>> > 12 May 2014 13:26:17,056 1245530 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 56 milli seconds for  100 payloads
>> > 12 May 2014 13:26:17,254 1245728 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 40 milli seconds for  38 payloads
>> > 12 May 2014 13:26:17,572 1246046 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 51 milli seconds for  100 payloads
>> > 12 May 2014 13:26:18,079 1246553 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 142 milli seconds for  100 payloads
>> > 12 May 2014 13:26:19,204 1247678 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 195 milli seconds for  100 payloads
>> > 12 May 2014 13:26:19,541 1248015 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 65 milli seconds for  100 payloads
>> > 12 May 2014 13:26:19,997 1248471 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 55 milli seconds for  100 payloads
>> > 12 May 2014 13:26:20,233 1248707 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 45 milli seconds for  100 payloads
>> > 12 May 2014 13:26:20,619 1249093 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 74 milli seconds for  100 payloads
>> > 12 May 2014 13:26:20,970 1249444 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 53 milli seconds for  100 payloads
>> > 12 May 2014 13:26:21,325 1249799 [zmqPub--2-thread-1] INFO  _LOGGER  -
>> > sendResponses took 103 milli seconds for  100 payloads
>> > 12 May 2014 13:26:21,651 1250125 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 57 milli seconds for  100 payloads
>> > 12 May 2014 13:26:21,913 1250387 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 40 milli seconds for  100 payloads
>> > 12 May 2014 13:26:22,276 1250750 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 66 milli seconds for  100 payloads
>> > 12 May 2014 13:26:22,692 1251166 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 202 milli seconds for  100 payloads
>> > 12 May 2014 13:26:22,775 1251249 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 13 milli seconds for  10 payloads
>> > 12 May 2014 13:26:23,083 1251557 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 53 milli seconds for  100 payloads
>> > 12 May 2014 13:26:23,461 1251935 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 66 milli seconds for  100 payloads
>> > 12 May 2014 13:26:24,032 1252506 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 164 milli seconds for  100 payloads
>> > 12 May 2014 13:26:25,112 1253586 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 272 milli seconds for  100 payloads
>> > 12 May 2014 13:26:25,510 1253984 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 73 milli seconds for  100 payloads
>> > 12 May 2014 13:26:26,345 1254819 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 83 milli seconds for  100 payloads
>> > 12 May 2014 13:26:26,789 1255263 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 60 milli seconds for  100 payloads
>> > 12 May 2014 13:26:26,848 1255322 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 9 milli seconds for  38 payloads
>> > 12 May 2014 13:26:27,421 1255895 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 78 milli seconds for  100 payloads
>> > 12 May 2014 13:26:27,787 1256261 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 75 milli seconds for  100 payloads
>> > 12 May 2014 13:26:28,600 1257074 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 111 milli seconds for  100 payloads
>> > 12 May 2014 13:26:29,090 1257564 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 71 milli seconds for  100 payloads
>> > 12 May 2014 13:26:29,465 1257939 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 71 milli seconds for  100 payloads
>> > 12 May 2014 13:26:29,808 1258282 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 44 milli seconds for  100 payloads
>> > 12 May 2014 13:26:30,172 1258646 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 57 milli seconds for  100 payloads
>> > 12 May 2014 13:26:30,397 1258871 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 46 milli seconds for  100 payloads
>> > 12 May 2014 13:26:30,785 1259259 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 148 milli seconds for  100 payloads
>> > 12 May 2014 13:26:31,025 1259499 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 45 milli seconds for  100 payloads
>> > 12 May 2014 13:26:31,298 1259772 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 45 milli seconds for  100 payloads
>> > 12 May 2014 13:26:31,644 1260118 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 66 milli seconds for  100 payloads
>> > 12 May 2014 13:26:31,804 1260278 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 31 milli seconds for  100 payloads
>> > 12 May 2014 13:26:31,841 1260315 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 7 milli seconds for  10 payloads
>> > 12 May 2014 13:26:32,750 1261224 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 184 milli seconds for  100 payloads
>> > 12 May 2014 13:26:33,343 1261817 [zmqPub--2-thread-1] INFO  LOGGER  -
>> > sendResponses took 114 milli seconds for  100 payloads
>> >
>> >
>> > Thanks
>> > Dileep.
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>
>



More information about the zeromq-dev mailing list