[zeromq-dev] ZMQ PUB server bottlenecks

Yanone post at yanone.de
Mon Nov 23 14:56:23 CET 2020


Hi everyone,

I hope you remember me, I asked about how to set up a scalable Pub/Sub server earlier as a migration from Google’s commercial Pub/Sub for my open source project.

With your help, by now I have a nice server humming along in Google Compute Engine, and I'm wondering where my bottlenecks are.

So far I’m watching 
- CPU
- memory
- total number of machine’s TCP connections (which are a few more than just the messaging ones).

I've tested 8,000 simultaneous connections from my computer, after which my home internet connection collapsed. So I have reason to believe that I can handle many more that Google’s artificial bottleneck of 10,000 topics (rather than connections).

The TCP connections are tied to file descriptors AFAIK. Two commands reveal them (both in the docker, not in the host):
- `ulimit -n` yields 1048576
- `cat /proc/sys/fs/file-max` yields 99773

So as far as I'm concerned, I can expect a bit less than 99773 maximum possible connections for this machine.
Is that correct, and is there anything else I need to watch?


Thanks for your help once again.
Jan



--

Yanone
Dipl.-Designer,
Master of Design in Type & Media
https://yanone.de
Twitter/Instagram: @yanone
+49 176 24023268 (Germany)

Type.World project:
https://type.world

Crowd funding on:
https://www.patreon.com/typeWorld




More information about the zeromq-dev mailing list