[zeromq-dev] Request Reply throughput

[jricher]

This is in the zmq book, but the long and the short of it is that you are bottlenecking on the send receive latency between the two nodes. It takes time for the second node to receive the frame from the first, process it, and send back the reply. During this time, the first node is idle. The first node is spending all its time idling waiting for acks, with lousy throughput. 
To fix this you need more frames outstanding. 
Jacques Richer602-350-2463
-------- Original message --------From: Jasper Jaspers <jaspers01995 at gmail.com> Date: 9/5/18  10:56 AM  (GMT-07:00) To: ZeroMQ development list <zeromq-dev at lists.zeromq.org> Subject: [zeromq-dev] Request Reply throughput 
In my setup I have two nodes connected via a 10 gigabit switch.  I ran iperf  and zeroMQ throughput (local_thr and remote_thr) to prove that I could achieve full bandwidth and I'm able to:iperf:   [ ID] Interval       Transfer     Bandwidth
   [  3]  0.0-10.0 sec  11.5 GBytes  9.90 Gbits/sec

zeroMQ: 
   message size: 500000 [B]
   message count: 10000
   mean throughput: 2474 [msg/s]
   mean throughput: 9896.000 [Mb/s]

I then wrote a simple zeroMQ request reply test application to measure the "throughput" using REQ/REP socket.  In this application the client runs on one node and sends N (10000) messages each of size M (500000) and waits for the reply for each message.  The server running on another node receives the messages and sends a simple acknowledgment reply message.  The client calculates the "throughput" based on how long it took to send all messages and receive all replies. I'm seeing that it takes about 11 seconds to complete which gives a throughput of 0.12 Gbit /sec.  Are these results expected due to the nature of the request reply sockets?  Should I expect to be anywhere near the full 10Gbit bandwidth with this test?

-Jasper









-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180905/1d956b6d/attachment.htm>