[zeromq-dev] zeromq / pub sub lost messages

[Jens Auer]

Hi,

 

how do you send the messages? The default HWM is 1000 messages and sending is done asynchronously in the IO thread. If you exceed this number of messages in the queue, e.g. by sending a batch of much more than 1000 messages you will experience message drop.

 

Best wishes,

Jens

 

Von: zeromq-dev [mailto:zeromq-dev-bounces at lists.zeromq.org] Im Auftrag von Philippe Guibert
Gesendet: Mittwoch, 4. Januar 2017 06:51
An: zeromq-dev at lists.zeromq.org
Betreff: [zeromq-dev] zeromq / pub sub lost messages

 

Hi all,

I am using zeromq 4.1.3 version through PUB/SUB operation.

Some information about the way it is implemented:

- It uses one single PUB application, and single SUB application, in separate process.

- Underlying Transport layer is IPC ( I am on the same machine).

- I checked that subscription is done far before the first publication message is sent.

- Message carried are 120 byte size.

- I use zmq_msg_recv() in reception and zmq_msg_send() in transmission.

- I check for return code on both sides.

I have one PUB application that sends several thousand of messages per second, and I expect that the SUB application will receive all messages.

The observation is that while the publisher sends 15000 messages, the subscriber can receive sometimes 15000, but other times 14750, or 11000. It does not seem to be deterministic.

I have no error codes related to the zmq_msg_recv() or zmq_msg_send() API.

Do you have any suggestions to give on how to handle that issue ?

Do you suggest to use pgm protocol instead of ipc ?

Are there any tracked issues related to that issue described ?

Thanks in advance for any suggestion,

Best Regards,

Philippe 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20170104/ca981c91/attachment.htm>