[zeromq-dev] Performance issues in multiple queues from a single process..
Arvind.N at citrix.com
Thu May 26 08:11:51 CEST 2011
We are stuck with a strange performance problem. The use case is this.. and queues are of type Q_TRANS_IPC for all Push/Pull queues.
First Case :::
Process1 ---------> Process2
(pushq) 2500 packets/sec (PullQ here, Receiving and processing all 2500 packets/sec no issues)
Note: Process2 has a request/response queue too. The request/Response queue and the PullQ have done their own zmq_init(1) and using it for themselves and no performance issues seen.
Process1 ---------------------------------------------> Process2 --------300 packets/sec --------------> Process3
(pushq) slows down gradually from 2500 (PullQ towards Process1, PushQ to process3) (PullQ towards Process2)
In the above usecase, we see performance issues ... first the moment, the call to zmq_send from Process2 is done, the performance of Process2 to pull comes down to almost half and even less as time progresses. This inturn leads to slowness in pulling. Commenting the zmq_send makes the Process2 makes the while thing work fine again.
Note: Process2 has a request/response queue too. The request/Response queue and the PullQ have done their own zmq_init(1) and the PushQ has done its own zmq_init(5) and using it. We thought maybe the issue is multiple zmq_init() .. hence make the PushQ and the PullQ share the same void* context of zmq_init() still we see the same performance issues. Also in both the cases the Request/Response queue is not handling any traffic at all, basically used for some control signaling at the startup.
Any ideas/thoughts most welcome.
More information about the zeromq-dev