[zeromq-dev] message loss at high send rate in Jyre

[Wynne Adam (CR/RTC3.1-NA)]

Hi All,

After talking with Pieter, I'm going to first do the test with a JNI wrapper for the zre lib.  If anyone has built such a wrapper and is willing to share, please let me know :)

Thanks
Adam 


-----Original Message-----
From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Utsav Drolia
Sent: Wednesday, September 02, 2015 10:17 AM
To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
Subject: Re: [zeromq-dev] message loss at high send rate in Jyre

Could you check where the messages do get dropped? 
Specifically, is there a buffer overflow at the pipe between the ZreInterface and the Agent? 

> On Sep 2, 2015, at 9:46 AM, Wynne Adam (CR/RTC3.1-NA) <Adam.Wynne at us.bosch.com> wrote:
> 
> I simplified a bit and now explicitly wait for all responders to JOIN but I'm getting the same results.  I can't see any real difference between your test and mine.  I think my test is correct unless I'm doing something really dumb that I can't see.  
> 
> Incidentally, if I lower the pause to 0, the ZreInterface.send() starts blocking, which I think is the behavior we'd expect.
> 
> We prefer to use the pure java version of Zyre so that we can use it in Android.  We'll use the C library for other platforms.
> 
> 
> Best regards
> 
> Adam Wynne
> CR/RTC3.1-NA
> 
> Tel. +1(412)390-3211
> 
> 
> -----Original Message-----
> From: zeromq-dev-bounces at lists.zeromq.org 
> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter 
> Hintjens
> Sent: Wednesday, September 02, 2015 3:30 AM
> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
> Subject: Re: [zeromq-dev] message loss at high send rate in Jyre
> 
> It might be a Jyre issue yet the most likely seems an issue in your test case (the loss is so dramatic). What I'd do is simplify it, and see when it starts happening.
> 
> If you are able to write a JNI layer then we could eventually wrap the Zyre C API in Java.
> 
> On Tue, Sep 1, 2015 at 11:10 PM, Wynne Adam (CR/RTC3.1-NA) <Adam.Wynne at us.bosch.com> wrote:
>> OK.  Do you think it's a problem in Jyre or JeroMQ?
>> 
>> 
>> Best regards
>> 
>> Adam Wynne
>> CR/RTC3.1-NA
>> 
>> Tel. +1(412)390-3211
>> 
>> 
>> -----Original Message-----
>> From: zeromq-dev-bounces at lists.zeromq.org
>> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter 
>> Hintjens
>> Sent: Tuesday, September 01, 2015 2:19 PM
>> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
>> Subject: Re: [zeromq-dev] message loss at high send rate in Jyre
>> 
>> So here's a C test case that does this:
>> https://gist.github.com/hintjens/d54d25c30e0339fb0a87
>> 
>> I do get message loss if I (a) start broadcasting before all nodes are ready or (b) don't wait for all messages to return. Otherwise it works even with no pause between message sends.
>> 
>> -Pieter
>> 
>> On Tue, Sep 1, 2015 at 11:25 AM, Pieter Hintjens <ph at imatix.com> wrote:
>>> Hi Adam,
>>> 
>>> I'm recreating this test case in C/Zyre to see how it performs.
>>> 
>>> -Pieter
>>> 
>>> On Mon, Aug 31, 2015 at 4:39 PM, Wynne Adam (CR/RTC3.1-NA) 
>>> <Adam.Wynne at us.bosch.com> wrote:
>>>> Hi Everyone,
>>>> 
>>>> 
>>>> 
>>>> We are evaluating Jyre for use in a project and are doing some 
>>>> benchmark testing.  We found that at higher send rates, some 
>>>> messages appear to get lost.  I have created a github project with 
>>>> multiple “nodes” running in their own threads to demonstrate the 
>>>> issue.  I included a README in the project that describes the test 
>>>> scenario and how to run it so it’s easy to run the test.
>>>> 
>>>> 
>>>> 
>>>> https://github.com/awynne/jyre-standalone-benchmark
>>>> 
>>>> 
>>>> 
>>>> Please let me know if you have any ideas and/or are able to look into it.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Best regards
>>>> 
>>>> Adam Wynne
>>>> 
>>>> Bosch Research and Technology Ctr.
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>> 
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev