[zeromq-dev] Reliable pub sub on c++ linux
Peter Krey
krey at ripple.com
Thu Apr 30 19:07:37 CEST 2015
i set the SND_HMW to 9000000 on the pub process and RCV_HMW to 9000000 on
the sub process fixed the issue.
On Wed, Apr 29, 2015 at 11:05 PM, Gerry Steele <gerry.steele at gmail.com>
wrote:
> That doesn't make sense. Did you set hwm on both sides of the connection?
> On 28 Apr 2015 17:26, "Peter Krey" <krey at ripple.com> wrote:
>
>> this fixed it so that the pub can hold up to a few seconds of throughput
>> in memory
>>
>> int hwm = 9000000;
>> publisher.setsockopt( ZMQ_SNDHWM, &hwm, sizeof (hwm));
>>
>> the documentation said that a hwm of zero would never flood; i think what
>> happened was memory couldn't be allocated fast enough. my test app is
>> sending a few million msgs/sec on the publisher.
>>
>>
>>
>> On Mon, Apr 27, 2015 at 1:08 PM, Peter Krey <krey at ripple.com> wrote:
>>
>>> I have HWM set to zero on recv and pub. I am keeping track of sequence
>>> numbers recved on the sub socket which are sent out by the pub socket. Here
>>> is an example output.
>>>
>>> The pub socket is publishing a uint64_t seqNumber. If i change the
>>> socket types to pair, no seqNumbers are ever missed.
>>>
>>>
>>> seqNumber missed 2301000
>>> seqNumber missed 2303206
>>> seqNumber missed 2305000
>>> seqNumber missed 2306820
>>> seqNumber missed 2309353
>>> seqNumber missed 2311575
>>> seqNumber missed 2314514
>>> seqNumber missed 2316767
>>> seqNumber missed 2318000
>>> seqNumber missed 2319924
>>> seqNumber missed 2321730
>>> seqNumber missed 2323618
>>> seqNumber missed 2325000
>>> seqNumber missed 2326963
>>> seqNumber missed 2329000
>>> seqNumber missed 2330664
>>> seqNumber missed 2333000
>>> seqNumber missed 2334997
>>> seqNumber missed 2336000
>>> seqNumber missed 2338000
>>> seqNumber missed 2340000
>>> seqNumber missed 2343000
>>> seqNumber missed 2344933
>>> seqNumber missed 2346401
>>> seqNumber missed 2349000
>>> seqNumber missed 2351000
>>> seqNumber missed 2352309
>>> seqNumber missed 2354198
>>> seqNumber missed 2356000
>>> seqNumber missed 2357645
>>>
>>> On Mon, Apr 27, 2015 at 12:56 PM, Pieter Hintjens <ph at imatix.com> wrote:
>>>
>>>> You can increase the HWM on sender and receiver to match your
>>>> expectations.
>>>>
>>>> If you set the HWM to zero there will never be any message loss, which
>>>> also means your publisher will explode if the subscriber stops
>>>> reading.
>>>>
>>>>
>>>>
>>>> On Mon, Apr 27, 2015 at 9:03 PM, Peter Krey <krey at ripple.com> wrote:
>>>> > Hi,
>>>> >
>>>> > What is the best way to get guaranteed in order delivery over pub-sub
>>>> > framework in zmq using c++ on linux?
>>>> >
>>>> > I have a test server and client running zmq pub and sub sockets. The
>>>> pub
>>>> > pushes sequence numbers as fast as possible in a tight loop. The sub
>>>> socket
>>>> > misses around one in every 10k messages.
>>>> >
>>>> > Thanks
>>>> >
>>>> > _______________________________________________
>>>> > zeromq-dev mailing list
>>>> > zeromq-dev at lists.zeromq.org
>>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>> >
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>
>>>
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150430/1acd097b/attachment.htm>
More information about the zeromq-dev
mailing list