[zeromq-dev] Inconsistent behavior during tests

[Joshua Foster]

The basic problem is that the remote is producing messages faster than the consumer (local). This is a common problem in messaging systems and if left unchecked will kill the queue. ZeroMQ has a few ways to help with the problem. The first is the HWM (High Water Mark). Once the HWM is reached, it will start throwing messages away. This is helpful in that the application won't die, but you need to have your application be able to handle missing data. If the data just comes in bursts, but the consumer will eventually be able to handle it, you can set the swap option. 0MQ will then start swapping the messages to disk up once the HWM is reached up to the allotted size. Of course, once the swap space is filled - it starts throwing messages away.

If you need to process more messages than the consumers handle, you need to start looking into sharding the data. You could do this by creating a simple load balancer device. I would strongly recommend reading the guide as it mentions a lot of this and more (http://zguide.zeromq.org/page:all).

I mostly have experience using the Java bindings and have not seen any of the inconsistent behavior. Since you are just looking at a simple throughput test, you may want to modify the remote_thr and put some sleeps in after a batch of messages are added. This will allow the client to throttle a little, but you will probably need to play with the numbers if you want to see the max. You will also notice that what bindings are used can make a difference in the performance. The C and C++ bindings will be the fastest. Python and Ruby will be the slowest. I don't think anyone has done benchmarks between the different bindings.

Joshua

On May 11, 2011, at 5:20 PM, Christian Martinez wrote:

> Hmm.. Are you using clrzmq2?
> 
> Built libzmq 2.1.6 by opening sln and just doing rebuild all.. System info below
>  
> OS Name             Microsoft Windows 7 Enterprise
> Version                6.1.7601 Service Pack 1 Build 7601
> System Manufacturer    Hewlett-Packard
> System Model   HP EliteBook 8540w
> System Type      x64-based PC
> Processor            Intel(R) Core(TM) i7 CPU       Q 720  @ 1.60GHz, 1600 Mhz, 4 Core(s), 8 Logical Processor(s)
> Installed Physical Memory (RAM)             16.0 GB
> Total Physical Memory  15.9 GB
> Available Physical Memory          11.1 GB
> Total Virtual Memory     31.9 GB
> Available Virtual Memory             26.3 GB
> Page File Space 15.9 GB
> Page File              C:\pagefile.sys
>  
> --CM
>  
> From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Joshua Foster
> Sent: Wednesday, May 11, 2011 2:06 PM
> To: ZeroMQ development list
> Subject: Re: [zeromq-dev] Inconsistent behavior during tests
>  
> I'm not sure what you are seeing here are my test runs...
> 
> E:\zmqwin32>remote_thr.exe tcp://127.0.0.1:5555 120 10000000
> 
> E:\zmqwin32>remote_thr.exe tcp://127.0.0.1:5555 120 10000000
> 
> E:\zmqwin32>local_thr.exe tcp://127.0.0.1:5555 120 10000000
> message size: 120 [B]
> message count: 10000000
> mean throughput: 752049 [msg/s]
> mean throughput: 721.967 [Mb/s]
> 
> E:\zmqwin32>local_thr.exe tcp://127.0.0.1:5555 120 10000000
> message size: 120 [B]
> message count: 10000000
> mean throughput: 748902 [msg/s]
> mean throughput: 718.946 [Mb/s]
> 
> I'm running on the local machine with Windows 7, Intel 2.8ghz quad core with 4GB of memory. I have 0MQ 2.1.6 compiled with VS2010. 
> 
> Joshua
> 
> On 5/11/2011 4:11 PM, Christian Martinez wrote:
> Running tests back to back on same laptop back to back sending 10000000 messages.
>  
> C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe tcp://127.0.0.1:5555 120 10000000
>  
> C:\software\zeromq-clrzmq2-f420936\remote_thr\bin\Release>remote_thr.exe tcp://127.0.0.1:5555 120 10000000
>  
> Unhandled Exception: ZMQ.Exception: Visual C++ CRT: Not enough memory to complete call to strerror.
>    at ZMQ.Socket.Send(Byte[] message, SendRecvOpt[] flags)
>   at remote_thr.Program.Main(String[] args)
>  
> Is this normal? If this condition is known to occur under load then how do I best modify the test to throttle or do whatever is necessary prevent the second result while still not killing throughput?
>  
> --CM
>  
>  
>  
>  
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>  
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110511/1daf8993/attachment.htm>