[zeromq-dev] ZMQ I/O threads CPU usage

[Francesco]

Hi all,

I found a sort of weird effect impacting greatly on the CPU usage of ZMQ
background threads. I found this behaviour with a very small benchmarking
utility I wrote using directly libzmq API, very small and self-contained (I
can share that on github if anyone's willing to take a look!)
This is 100% reproducible, at least on my Centos7 machine.

Here's the thing: the app with --pub CLI option starts a PUB server and
--sub starts a SUB server. I run the pub on a server connected by a 1Gbps
link to the sub server.
If in the pub server the logic is:

set ZMQ_XPUB_NODROP on the pub_skt
while (true)
   zmq_msg_send(dummy_msg, pub_skt, 0);
   // no wait of any kind

I measure around 940Mbps of throughput and the cpu usage of my application
thread is just 8% and zmq background thread is just 12% of cpu usage.
Wonderful.

Now if I change the logic to be:

set ZMQ_XPUB_NODROP on the pub_skt
while (true) {
   zmq_msg_send(dummy_msg, pub_skt, 0);
   spin_loop(2.5usec)
}

I measure a throughput of 700Mbps (expected due to the spin_loop that
simulates the time it takes a real application to produce a msg) and my
application thread goes to 100%... that's fine due to the spin_loop.
However the strange thing is that the ZMQ background thread also tops at
97% of CPU usage !!!

So in first scenario the zmq background thread used only 12% of cpu to fill
914Mbps ; in second scenario it uses 97% to fill 700Mbps...

how's that possible?

Any help GREATLY appreciated

Thanks!!

Francesco
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20210401/8cb525bd/attachment.htm>