[zeromq-dev] Performance issues in multiple queues from a single process..

[Arvind N]

Hi,
     We are stuck with a strange performance problem. The use case is this.. and queues are of type Q_TRANS_IPC for all Push/Pull queues. 

First Case ::: 

Process1                     --------->                     Process2                                                                      
 (pushq)                      2500  packets/sec     (PullQ here, Receiving and processing all 2500 packets/sec no issues)

Note: Process2 has a request/response queue too. The request/Response queue and the PullQ have done their own zmq_init(1) and using it for themselves and no performance issues seen.

Second Case:::

Process1                          --------------------------------------------->                      Process2                                             --------300 packets/sec -------------->            Process3                                        
 (pushq)                                   slows down gradually from 2500                   (PullQ towards Process1, PushQ to process3)                                                       (PullQ towards Process2)

In the above usecase, we see performance issues ... first the moment, the call to zmq_send from Process2  is done, the performance of Process2 to pull comes down to almost half and even less as time progresses. This inturn leads to slowness in pulling. Commenting the zmq_send makes the Process2 makes the while thing work fine again. 

Note: Process2 has a request/response queue too. The request/Response queue and the PullQ have done their own zmq_init(1) and the PushQ has done its own zmq_init(5) and using it. We thought maybe the issue is multiple zmq_init() .. hence make the PushQ and the PullQ share the same void* context of zmq_init() still we see the same performance issues.  Also in both the cases the Request/Response queue is not handling any traffic at all, basically used for some control signaling at the startup. 

Any ideas/thoughts most welcome. 

Thanks
Arvind