[zeromq-dev] zeromq / pub sub lost messages

Philippe Guibert philippe.guibert at 6wind.com
Wed Jan 4 10:43:50 CET 2017


On Wed, Jan 4, 2017 at 10:34 AM, Jens Auer <jens.auer at betaversion.net> wrote:

Hello Jens,

Best wishes too,

> Hi,
>
>
>
> how do you send the messages?

I use zmq_msg_send() procedure.
I also used zmq_send() procedure with and without MSG_NO_WAIT flag.
I expect that return code sends me an error when dropping message,
which is not the case.

>The default HWM is 1000 messages and sending

I used tcp transport connection, and wireshark. I observed that the
packets number sent was matching the received messages on the
subscriber side.
I deduced the problem was due to an emission problem, not a reception problem.

I did a counter test by setting ZMQ_SNDHWM to a big value ( in my case 60000).
I could observe no drops with both ipc and tcp transport connection.

> is done asynchronously in the IO thread. If you exceed this number of
> messages in the queue, e.g. by sending a batch of much more than 1000
> messages you will experience message drop.

How can I know there are some drop messages ?
If I make a limitation, I expect to handle the flow congestion, in
order to resubmit to publisher, but it seems publisher does not return
me valuable error code.

Best Regards,

Philippe

>
>
>
> Best wishes,
>
> Jens
>
>
>
> Von: zeromq-dev [mailto:zeromq-dev-bounces at lists.zeromq.org] Im Auftrag von
> Philippe Guibert
> Gesendet: Mittwoch, 4. Januar 2017 06:51
> An: zeromq-dev at lists.zeromq.org
> Betreff: [zeromq-dev] zeromq / pub sub lost messages
>
>
>
> Hi all,
>
> I am using zeromq 4.1.3 version through PUB/SUB operation.
>
> Some information about the way it is implemented:
>
> - It uses one single PUB application, and single SUB application, in
> separate process.
>
> - Underlying Transport layer is IPC ( I am on the same machine).
>
> - I checked that subscription is done far before the first publication
> message is sent.
>
> - Message carried are 120 byte size.
>
> - I use zmq_msg_recv() in reception and zmq_msg_send() in transmission.
>
> - I check for return code on both sides.
>
> I have one PUB application that sends several thousand of messages per
> second, and I expect that the SUB application will receive all messages.
>
> The observation is that while the publisher sends 15000 messages, the
> subscriber can receive sometimes 15000, but other times 14750, or 11000. It
> does not seem to be deterministic.
>
> I have no error codes related to the zmq_msg_recv() or zmq_msg_send() API.
>
> Do you have any suggestions to give on how to handle that issue ?
>
> Do you suggest to use pgm protocol instead of ipc ?
>
> Are there any tracked issues related to that issue described ?
>
> Thanks in advance for any suggestion,
>
> Best Regards,
>
> Philippe
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev



More information about the zeromq-dev mailing list