[zeromq-dev] performance issue in zmq_proxy_steerable()

[Claudio U.]

Hi all,

I am facing a performance issue with a thread running zmq_proxy_steerable(), which subscribes to multiple inproc XPUB sockets and forwards data to a tcp XPUB socket.
I am using ZMQ latest version 4.3.1

I noticed a huge amount of calls to poll() that limit the throughput to ~120kpps. From strace output I see:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
81.77    1.419731           4    371984           poll
18.22    0.316395           7     45533           write
 0.01    0.000106           5        22           read
 0.00    0.000015          15         1           restart_syscall
------ ----------- ----------- --------- --------- ----------------
100.00    1.736247                417540           total


If I use gdb to stop on the poll() it seems that most of the time it is called by:

zmq::socket_base_t::getsockopt(...)

that calls process_command() with flag throttle_=false :

  const int rc = process_commands (0, false); // socket_base.cpp:450
  // prototype: int zmq::socket_base_t::process_commands (int timeout_, bool throttle_)

I tried to change that line to throttle=true and to tune the parameters in config.hpp :
  inbound_poll_rate = 100 -> inbound_poll_rate = 10000
  max_command_delay = 3000000 -> max_command_delay  = 3000000000


The result is an increase of the throughput from ~120kpps to ~230kpps due to the lower amount of calls to poll():

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
55.64    1.278789           4    295038           poll
44.35    1.019367           7    147408           write
 0.00    0.000086           4        22           read
 0.00    0.000057          10         6           futex
------ ----------- ----------- --------- --------- ----------------
100.00    2.298299                442474           total
 
Is it safe to enable the throttle in getsockopt()?
Is there any side effect beside a maybe longer response time to commands?
In case it is safe, do you accept a PR to change throttle=true and make inbound_poll_rate and max_command_delay configurable?


Regards,
Claudio