[zeromq-dev] Zeromq messages getting dropped

[Ritesh Adval]

Hi,

I have been using zeromq (2.2.0) with java binding which is used by storm (
https://github.com/zeromq/jzmq).
Though I am using jzmq binding from storm, I think this is a zeromq bug. I
have a JAVA junit test which reproduce this error. I can send it to someone
who can help look into this issue.

My set up is as follows:

environment: Gentoo Linux

riteshadval at aw26 ~/work/source/git/cloud/zmq-dropped-msg $ uname -a
Linux aw26 3.5.3-gentoo #4 SMP Thu Oct 4 03:35:09 UTC 2012 x86_64 Intel(R)
Core(TM) i7 CPU 960 @ 3.20GHz GenuineIntel GNU/Linux


(1) I have a broker with router and dealer socket and it uses polling to
transfer messages among sockets. I started broker which binds these two
sockets on some port.
(2) I have a test program in JAVA where I start 100 concurrent threads,
each sends 100 message (approximate size of each message is 17kb) using a
dealer socket to broker's router socket. (I made sure sockets are not
shared among threads and using one parent zmq context, the hwm and linger
on all sockets is default value, ie. hwm is 0 and linger is very large
number)
(3) I connect to broker's dealer socket using a dealer socket and count
messages. I expect total count to be 10000.
(4) What I am noticing is that I do not get 10000 message, I am seeing that
LAST MESSAGE in some of my test program threads are getting dropped. I get
9996 or some other number messages. Usually 2-20 messages are getting
dropped.


I found that people have reported this error and one suggestion is to put
sleep after sending message. If I put sleep after sending message using a
socket it does work but not always, specially under large message size and
heavy load.



Anyone has idea on what could be the issue?

Thanks
Ritesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130116/bba04103/attachment.htm>