[zeromq-dev] 10 seconds delay on a PUB/SUB

Giacomo Tesio giacomo at tesio.it
Sat Mar 15 01:09:37 CET 2014


Thanks for your feedback.

The heavy load connection (PUB1) publish 50-100 msg/s (with few daily peaks
at 1000 msg/s). Such messages are between 50 bytes and 200 bytes in size
(almost 3/4 of messages are 200 bytes long).
The low load connection (PUB2) publish 50-500 msg/s with an average size of
200 bytes.

I have exactly 2 subscribers for PUB1 and 1 subscriber for PUB2.

As for the latency, all connections are local (127.0.0.1, with a port for
PUB1 and another for PUB2). I would use a simpler protocol than tcp, since
all I need is an ipc channel, but I'm on windows 7.


Giacomo
PS: on HWM and pub/sub: I'm not an expert, but it looks reliable from my
tests. Isn't it?



On Fri, Mar 14, 2014 at 7:43 PM, Gerry Steele <gerry.steele at gmail.com>wrote:

> On a tangent... Does high watermark=0 really make pub/ sub fully reliable?
> Wasn't my understanding. Could be wrong.
>
> How big are the messages you are sending?
>
> Can you reproduce on same hardware with a hello world pub sub for messages
> of the same size?
>  On 14 Mar 2014 15:06, "Giacomo Tesio" <giacomo at tesio.it> wrote:
>
>> Hi, I'm getting 5 to 10 seconds delay in on a pub/sub socket with low
>> load (in a context with heavy load on other sockets).
>>
>> I'm using NetMQ on Windows 7, with tcp transport on 127.0.0.1 (indeed it
>> should be ipc, but it's not supported on Windows AFAIK).
>>
>> This is the topology:
>>
>> We have Server A, Client B and Client C.
>>
>> Server binds a PUB with heavy load (let's call it PUB1), publishing
>> 50-100 msg/s with few daily peaks at 1000 msg/s.
>>
>> Server binds a PUB with small load (let's call it PUB2). publishing
>> 50-500 msg *each day*. Note however that these messages are sent in
>> groups of 1 to 5 in a few milliseconds.
>>
>> Client B connect with a SUB socket to PUB1,
>> Client C connects with two SUB socket to PUB1 and PUB2.
>>
>> My issue is that when a group of messages is sent in PUB2, the first is
>> received almost instantly from Client C, but the others are received
>> seconds after, at seconds of distances.
>>
>> For example, here are a few times from today problems.
>>
>> Sent from Server A  -> Received from Client C
>> 09:00:59.608 -> 09:01:05.643
>> 09:01:00.055 -> 09:01:05.64
>> 09:01:00.117 -> 09:01:10.928
>> 09:01:02.883 -> 09:01:16.172
>> 09:01:05.541 -> 09:01:18.754
>>
>>
>> How can I reduce this delay?
>> I tried to increase the ThreadPoolSize up to the number of CPUs, but
>> without success.
>> Note that I (must) have HighWaterMark = 0 on every socket (I can't loose
>> messages), but the machine is full of free memory (4 GB are always free)
>> and never use more than 40% of each cpu.
>>
>>
>> Giacomo
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140315/2fbe3a48/attachment.htm>


More information about the zeromq-dev mailing list