[zeromq-dev] pub-sub losing messages

[Michael Keselman]

I tries to increase HWM and also removed it altogether. It is still missing
messages and results are really inconstant. There is one subscriber sending
100,000 messages. With one subscriber, in 3 separate runs
the subscriber received respectively 76005, 37488, 82121 messages  For 3
subscribers, the numbers were  34778/34793/35535, 42152/42549/58702,
34375/38905/37519 per subscriber per run.

I am running all processes on a single VM with no other applications running


On Tue, May 21, 2013 at 10:02 AM, Pieter Hintjens <ph at imatix.com> wrote:

> On pub-sub sockets, when you exceed the HWM, messages are dropped. (This
> is the only sane strategy to avoid memory exhaustion when subscribers run
> too slowly.)
>
> You're sending 100K messages with a HWM of 1,000, so about 99% of the
> messages are going to be dropped.
>
> If the scenario of sending that many messages in one go is realistic, then
> raise the HWM to suit.
>
> -Pieter
>
>
> On Tue, May 21, 2013 at 6:50 PM, Michael Keselman <
> michael.keselman at gmail.com> wrote:
>
>> I am playing ZMQ using Java. I created very basic pub/sub application
>> (see below) with subscriber emulating slow response via random delay.
>> Unfortunately something isn't working. Whether I run multiple subscribers
>> or only one, they have never received all messages. I thought that setting
>> high water mark would help, but it didn't. Am I doing something wrong or
>> missing some nuance?
>>
>> -----Pub-----
>>
>> import org.zeromq.ZMQ;
>>
>> public class Pub {
>>         public static void main(String[] args) {
>>                 ZMQ.Context context = ZMQ.context(1);
>>         ZMQ.Socket publisher = context.socket(ZMQ.PUB);
>>         publisher.setHWM(1000);
>>         publisher.bind("tcp://*:5555");
>>
>>         long counter = 0;
>>                 String msg;
>>                 for (int i=0; i<100000; i++){
>>                 msg = Long.toString(counter)+"*******";
>>                 publisher.send(msg , 0);
>>                         System.out.println(msg);
>>                         counter++;
>>         }
>>
>>         publisher.close ();
>>         context.term ();
>>         }
>>
>> }
>>
>> -----Sub-----
>>
>> import org.zeromq.ZMQ;
>> import java.util.Random;
>>
>> public class sub {
>>         public static void main(String[] args) throws Exception{
>>                 Random rand = new Random();
>>                 ZMQ.Context context = ZMQ.context(1);
>>         ZMQ.Socket subscriber = context.socket(ZMQ.SUB);
>>         subscriber.connect("tcp://localhost:5555");
>>                 subscriber.subscribe("".getBytes());
>>
>>                 String msg;
>>         while (1>0){
>>             msg = subscriber.recvStr(0);
>>             System.out.println(msg);
>>                         Thread.sleep(rand.nextInt(3));
>>         }
>>         //subscriber.close();
>>         //context.term();
>>         }
>> }
>>
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>


-- 
-Michael Keselman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130521/d03245ad/attachment.htm>