[zeromq-dev] Only first X messages received with PGM/0MQ c++ setup

[Luca Boccassi]

On Mon, 2018-04-16 at 18:25 +0200, Johnny Berentsen wrote:
> 2018-04-16 14:35 GMT+02:00 Luca Boccassi <luca.boccassi at gmail.com>:
> 
> > On Mon, 2018-04-16 at 13:54 +0200, Johnny Berentsen wrote:
> > > 2018-04-02 15:30 GMT+02:00 Johnny Berentsen <johnny.berentsen at gma
> > > il.c
> > > om>:
> > > 
> > > > Hi all
> > > > <snip initial explanation>
> > > > 
> > > 
> > > Now I have a vector with size 2500 of std::strings filled with
> > > 4096 #
> > > characters. Every ten seconds I loop through the vector and send
> > > the
> > > data
> > > over a EPGM socket. If I go full throttle, only the first 1000
> > > messages are
> > > received, but if I add a usleep(1) after each send, all messages
> > > are
> > > received.
> > 
> > Default high water mark is 1000 messages, most likely the receiving
> > side is too slow to process and so it starts dropping messages
> > 
> > 
> 
> You're absolutely right. I was put off by the documentation that
> said:
> The default *ZMQ_SNDHWM* value of zero means "no limit".
> 
> Shame on me for not trying though :/
> 
> I understand that setting the HWM is different depending on the usage
> pattern. However, I can't find much information on the different
> criterias
> that should be considered.
> I've come up with these:
> Typical and max packet size, network restrictions, number of packets,
> available memory, CPU
> 
> My application will run on a LAN on modern PC's with >=8GiB memory,
> up to a
> few kB's per message and only a burst of a few thousand messages once
> in a
> while. What value is reasonable in such a case?

It mostly depend on your application and how quickly it can pull
messages off the socket. So the only way is to try and test it.

> I also tested with checking the return value of send() but it
> returned true
> regardless of sending or not. Are there other ways of being noticed
> that
> messages have been dropped?

No, that's the whole point about pub-sub - if you don't want a lossy
mechanism then you should use a different pattern.
Or you can set unlimited water marks and deal with the unbounded memory
usage.

-- 
Kind regards,
Luca Boccassi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180416/476562ac/attachment.sig>