[zeromq-dev] PUB-SUB pattern, ZeroMQ drops 32768 messages when 4M messages/sec are sent.
kangmo.kim at thankyousoft.com
Sun Sep 16 23:54:05 CEST 2012
I am developing LLDB(low latency database) which replicates transactional
log from a master to multiple slaves.
Using zeromq, I am pretty much satisfied with it except it drops messages
when I send 4 million messages / sec.
The pattern I use :
PUB - SUB.
I have 1 PUB, and 1 SUB process in my test.
The test environment :
4 core, 8 GB ram, windows 7 x64.
Release Mode, compilation done with Visual C++ 2010 express.
What PUB does :
Pub sends 4 million messages per second to SUB.
What SUB does :
Sub should receive 4 million messages per second but, after receiving about
50K messages, exactly 32768 messages are dropped.
I am checking with a sequence number assigned by PUB. it starts from 1,
increases monotonously without any hole nor dup.
Both PUB and SUB uses the default value 0 (unlimitted). I double checked it
When PUB sends huge amount of messages such as 4 million messages per
second, can messages be dropped even though ZMQ_HWM is set to 0?
I made sure that only half of 8GB RAM was used by the system when messages
are dropped, so I am assuming this is not about out of memory exceptional
In which case ZMQ drops exactly 32768 messages? I see sequence number in
SUB is changging for example, from 50000 to 82768(50000+32768).
Is there any log that I can check who dropped messages(PUB? SUB?) and why
it was dropped?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the zeromq-dev