[zeromq-dev] Zeromq messages getting dropped

Ritesh Adval riteshadval at gaikai.com
Sun Jan 27 20:06:20 CET 2013


Hi Min,

This is great, thanks for looking into it.
I am looking forward to trying out the fix.

-Ritesh
Sent from my iPhone.


On Jan 27, 2013, at 9:16 AM, Yu Dongmin <miniway at gmail.com> wrote:

> Hi,
> 
> I've found a culprit which caused the data loss. 
> 
> When ZMQ send a large message, the stream_engine sends data through multiple out_event calls.
> 
> The ZMQ linger option only guarantees messages are delivered to the peer pipe. By the speculative write out_event is called at least once but large message requires multiple hops.
> 
> Before finishing enough out_event calls, stream_engine can be terminated.
> 
> 
> So a longer linger option will not resolve this issue. A workaround seems to be adding some sleeps before close.  
> 
> 
> I'm going to submit a pull request to resolving the issue.
> 
> Thanks
> Min
> 
> On Jan 27, 2013, at 12:42 AM, Yu Dongmin <miniway at gmail.com> wrote:
> 
>> Hi,
>> 
>> My guess was it might have an issue on libzmq (zeromq c library) when large messages were heavily sent.
>> 
>> Thanks
>> Min
>> 
>> On Jan 26, 2013, at 4:01 PM, Ritesh Adval <riteshadval at gaikai.com> wrote:
>> 
>>> Hi Min,
>>> 
>>> Thanks for the update.Just to confirm, 
>>> Are you saying that this issue is on zeromq c library or jzmq c wrapper?
>>> 
>>> Just an update that when I replaced 
>>> DEALER socket which connects to ROUTER socket of broker with REQ socket and replaced DEALER socket which connects to DEALER socket of broker with REP socket, then I do not see message loss when doing the same test. (REQ socket does "send" and then "recv" and REP does opposite  "recv" and "send")
>>> 
>>> -Ritesh
>>> Sent from my iPhone.
>>> 
>>> 
>>> On Jan 25, 2013, at 8:42 PM, Min <miniway at gmail.com> wrote:
>>> 
>>>> I was able to reproduce the issue on jzmq even on zeromq 3.2.2.
>>>> 
>>>> What I discovered is about last 30K bytes of 45K message was not sometimes delivered to in-router on raw close. 
>>>> I didn't build equivalent C code, as jzmq is a thin wrapper of native C library it could have the same problem.
>>>> 
>>>> But I didn't find a clear solution yet.
>>>> 
>>>> Thanks
>>>> Min
>>>> 
>>>> 
>>>> On Thu, Jan 24, 2013 at 6:39 AM, Ritesh Adval <riteshadval at gaikai.com> wrote:
>>>> Hello,
>>>> 
>>>> I have created a bug for this issue with instructions and java test case. Its at https://zeromq.jira.com/browse/LIBZMQ-497
>>>> 
>>>> Thanks
>>>> Ritesh
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Tue, Jan 22, 2013 at 6:30 PM, Ritesh Adval <riteshadval at gaikai.com> wrote:
>>>> Thanks Min,
>>>> 
>>>> I will create a bug with instruction and unit test. I was also experimenting with Java only version of zeromq (https://github.com/zeromq/jeromq). When running same test it does not drop message but has some other issue.
>>>> 
>>>> -Ritesh
>>>> 
>>>> 
>>>> 
>>>> On Mon, Jan 21, 2013 at 11:53 PM, Min <miniway at gmail.com> wrote:
>>>> Ritesh,
>>>> 
>>>> If you can reproduce the problem, Java code should be fine.
>>>> 
>>>> Community could look into it.
>>>> 
>>>> Thanks
>>>> Min
>>>> 
>>>> 2013년 1월 17일 목요일에 Ritesh Adval님이 작성:
>>>> 
>>>> Hi Charles,
>>>> 
>>>> I have test program in JAVA, I am not a C programmer so i will probably take me time to reproduce this in C. Can someone first take a look at my JAVA program to see if I am not doing anything stupid.  Should I create bug and attach Java maven project?
>>>> Its very easy to run it, all you need is zeromq 2.2.0 installed and jzmq built and installed by building jzmq (https://github.com/zeromq/jzmq).
>>>> I can add instructions to the bug report. Once confirmed that program  looks right I can try to create a C version of the test but will take me some time.
>>>> 
>>>> let me know.
>>>> 
>>>> Thanks
>>>> Ritesh
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Wed, Jan 16, 2013 at 10:55 PM, Charles Remes <lists at chuckremes.com> wrote:
>>>> On Jan 16, 2013, at 4:08 PM, Ritesh Adval <riteshadval at gaikai.com> wrote:
>>>> 
>>>> > Hi Charles,
>>>> >
>>>> > Yes I close the socket in my thread after sending 100 messages, and I expect that LINGER will make sure messages are sent to the other end, I expected that context termination will block and make sure any pending messages are sent, but thats not happening. context termination returns quickly.
>>>> >
>>>> > Just now tried again in my unit test by setting LINGER to Integer.MAX_VALUE explicitly in all my sockets and ran the test again and it did fail with messages getting dropped.
>>>> >
>>>> > The interesting thing is only the 100th message  (The last one) from some of my concurrent threads are getting dropped.
>>>> 
>>>> Time to show someone the code. That's the easiest way to figure it out. If you can reproduce this in C, that will get a lot more attention.
>>>> 
>>>> Here's how to open an issue:
>>>> 
>>>> http://www.zeromq.org/docs:issue-tracking
>>>> 
>>>> cr
>>>> 
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>> 
>>>> 
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>> 
>>>> 
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130127/201c1c73/attachment.htm>


More information about the zeromq-dev mailing list