[zeromq-dev] message loss at high send rate in Jyre

Pieter Hintjens ph at imatix.com
Thu Sep 3 17:30:53 CEST 2015


Done. I wasn't watching jzmq, am again now.

On Thu, Sep 3, 2015 at 5:19 PM, Trevor Bernard <trevor.bernard at gmail.com> wrote:
> Ok, can you merge this pull request please.
> https://github.com/zeromq/jzmq/pull/387
>
> On Thu, Sep 3, 2015 at 12:17 PM, Pieter Hintjens <ph at imatix.com> wrote:
>> Trevor,
>>
>> I see from http://zeromq.org/docs:tuning-zeromq#toc1 that OS/X has a
>> low default limit on file handles. Could be this. Zyre test cases tend
>> to consume a lot of file handles...
>>
>> -Pieter
>>
>> On Thu, Sep 3, 2015 at 5:09 PM, Trevor Bernard <trevor.bernard at gmail.com> wrote:
>>> Adam,
>>>
>>> I was able to reliably recreate your error using JeroMQ on my MBP
>>> though I have yet to try on a Linux. When I did use JZMQ with
>>> libzmq-4.1.3, I would get: Resource temporarily unavailable
>>> (src/signaler.cpp:301) after a certain point in time. I'm wondering if
>>> this is mac specific? I'll set up a VM and retry the tests.
>>>
>>> -Trev
>>>
>>> On Thu, Sep 3, 2015 at 11:59 AM, Trevor Bernard
>>> <trevor.bernard at gmail.com> wrote:
>>>> Hi Adam,
>>>>
>>>> Building JZMQ is pretty straight forward:
>>>>
>>>> $ ./autogen.sh && ./configure && make && sudo ldconfig && mvn install
>>>> -Dgpg.skip=true
>>>>
>>>> I submitted a few patches to jyre this morning that made jeromq the
>>>> default. Also, if you want to test with JZMQ, you can just use the
>>>> follow branch in my fork:
>>>> https://github.com/trevorbernard/jyre/tree/jzmq
>>>>
>>>> You will need to mvn install jyre as well.
>>>>
>>>> I made some changes to your test here:
>>>> https://github.com/trevorbernard/jyre-standalone-benchmark
>>>>
>>>> I made the counters thread safe and I changed the gradle build to look
>>>> for artifacts in ~/.m2/repository
>>>>
>>>> Let me know how your testing goes.
>>>>
>>>> -Trev
>>>>
>>>> On Thu, Sep 3, 2015 at 9:19 AM, Wynne Adam (CR/RTC3.1-NA)
>>>> <Adam.Wynne at us.bosch.com> wrote:
>>>>> Hi All,
>>>>>
>>>>> After talking with Pieter, I'm going to first do the test with a JNI wrapper for the zre lib.  If anyone has built such a wrapper and is willing to share, please let me know :)
>>>>>
>>>>> Thanks
>>>>> Adam
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Utsav Drolia
>>>>> Sent: Wednesday, September 02, 2015 10:17 AM
>>>>> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
>>>>> Subject: Re: [zeromq-dev] message loss at high send rate in Jyre
>>>>>
>>>>> Could you check where the messages do get dropped?
>>>>> Specifically, is there a buffer overflow at the pipe between the ZreInterface and the Agent?
>>>>>
>>>>>> On Sep 2, 2015, at 9:46 AM, Wynne Adam (CR/RTC3.1-NA) <Adam.Wynne at us.bosch.com> wrote:
>>>>>>
>>>>>> I simplified a bit and now explicitly wait for all responders to JOIN but I'm getting the same results.  I can't see any real difference between your test and mine.  I think my test is correct unless I'm doing something really dumb that I can't see.
>>>>>>
>>>>>> Incidentally, if I lower the pause to 0, the ZreInterface.send() starts blocking, which I think is the behavior we'd expect.
>>>>>>
>>>>>> We prefer to use the pure java version of Zyre so that we can use it in Android.  We'll use the C library for other platforms.
>>>>>>
>>>>>>
>>>>>> Best regards
>>>>>>
>>>>>> Adam Wynne
>>>>>> CR/RTC3.1-NA
>>>>>>
>>>>>> Tel. +1(412)390-3211
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: zeromq-dev-bounces at lists.zeromq.org
>>>>>> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter
>>>>>> Hintjens
>>>>>> Sent: Wednesday, September 02, 2015 3:30 AM
>>>>>> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
>>>>>> Subject: Re: [zeromq-dev] message loss at high send rate in Jyre
>>>>>>
>>>>>> It might be a Jyre issue yet the most likely seems an issue in your test case (the loss is so dramatic). What I'd do is simplify it, and see when it starts happening.
>>>>>>
>>>>>> If you are able to write a JNI layer then we could eventually wrap the Zyre C API in Java.
>>>>>>
>>>>>> On Tue, Sep 1, 2015 at 11:10 PM, Wynne Adam (CR/RTC3.1-NA) <Adam.Wynne at us.bosch.com> wrote:
>>>>>>> OK.  Do you think it's a problem in Jyre or JeroMQ?
>>>>>>>
>>>>>>>
>>>>>>> Best regards
>>>>>>>
>>>>>>> Adam Wynne
>>>>>>> CR/RTC3.1-NA
>>>>>>>
>>>>>>> Tel. +1(412)390-3211
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: zeromq-dev-bounces at lists.zeromq.org
>>>>>>> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter
>>>>>>> Hintjens
>>>>>>> Sent: Tuesday, September 01, 2015 2:19 PM
>>>>>>> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
>>>>>>> Subject: Re: [zeromq-dev] message loss at high send rate in Jyre
>>>>>>>
>>>>>>> So here's a C test case that does this:
>>>>>>> https://gist.github.com/hintjens/d54d25c30e0339fb0a87
>>>>>>>
>>>>>>> I do get message loss if I (a) start broadcasting before all nodes are ready or (b) don't wait for all messages to return. Otherwise it works even with no pause between message sends.
>>>>>>>
>>>>>>> -Pieter
>>>>>>>
>>>>>>> On Tue, Sep 1, 2015 at 11:25 AM, Pieter Hintjens <ph at imatix.com> wrote:
>>>>>>>> Hi Adam,
>>>>>>>>
>>>>>>>> I'm recreating this test case in C/Zyre to see how it performs.
>>>>>>>>
>>>>>>>> -Pieter
>>>>>>>>
>>>>>>>> On Mon, Aug 31, 2015 at 4:39 PM, Wynne Adam (CR/RTC3.1-NA)
>>>>>>>> <Adam.Wynne at us.bosch.com> wrote:
>>>>>>>>> Hi Everyone,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> We are evaluating Jyre for use in a project and are doing some
>>>>>>>>> benchmark testing.  We found that at higher send rates, some
>>>>>>>>> messages appear to get lost.  I have created a github project with
>>>>>>>>> multiple “nodes” running in their own threads to demonstrate the
>>>>>>>>> issue.  I included a README in the project that describes the test
>>>>>>>>> scenario and how to run it so it’s easy to run the test.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://github.com/awynne/jyre-standalone-benchmark
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please let me know if you have any ideas and/or are able to look into it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>>
>>>>>>>>> Adam Wynne
>>>>>>>>>
>>>>>>>>> Bosch Research and Technology Ctr.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> zeromq-dev mailing list
>>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> zeromq-dev mailing list
>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>> _______________________________________________
>>>>>>> zeromq-dev mailing list
>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>> _______________________________________________
>>>>>> zeromq-dev mailing list
>>>>>> zeromq-dev at lists.zeromq.org
>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>> _______________________________________________
>>>>>> zeromq-dev mailing list
>>>>>> zeromq-dev at lists.zeromq.org
>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev



More information about the zeromq-dev mailing list