[zeromq-dev] Can I prevent zeromq from occupying file descriptors?

[Doron Somech]

It is not zeromq, its TCP, when you have 10K open connections you are using
10K file descriptors, there is no wonder here, that how TCP works. HAProxy
and nginx for example do the same, or any webserver, if you want to support
10K concurrent connections you must increase the maximum sockets per
application.

Take a look a the accept socket docs:

http://man7.org/linux/man-pages/man2/accept.2.html

On Tue, Jul 19, 2016 at 7:12 PM, Alexander Poddey <alexander.poddey at gmx.net>
wrote:

> Hmm,
>
> on linux, the number of file descriptors per process is limited (typically
> to 1024). The total limit on my machine (kernel limit is ~734 000 opened
> files).
>
> This means a process can accept only 1024 connections?!?
>
> I can tweak the limit as super user of that machine; this however is no
> good
> solution.
> Think of my agent simulation running on a server where I'm not root. Then
> each process can accept only 1024 connections which is way to few...
>
> I wonder why zmq needs a file descriptor per connection...
>
> Alexander
>
>
>
> Doron Somech wrote:
>
> > I'm  not sure I  understand,  each file descriptor is a socket.  When
> > zeromq accept tcp socket another file descriptor is created. OS should be
> > able to manage thousands and tens of thousand open descriptor.
> >
> > On Jul 19, 2016 6:56 PM, "Alexander Poddey" <alexander.poddey at gmx.net>
> > wrote:
> >
> >>
> >> Essentially, the question is:
> >> why does ZMQ consume file descriptors at all?
> >>
> >>
> >>
> >>
> >> > Hi,
> >> >
> >> > I have a setup in C++ on linux where a master process is setting up a
> >> > ZMQ_ROUTER and then forks many child processes, which then connect to
> >> that
> >> > router (tcp protocol).
> >> > Whenever a child zmq_connect's to the master, a file descriptor is
> >> > occupied (and assigned to the master process). They get free'ed fine
> >> > when the connection is close.
> >> >
> >> > This however limits the number of interacting processes to the number
> >> > of allowed file descriptors (per process). For me (linux), this
> >> > currently is 1024, which is way to small for my intended use
> >> > (multi-agent / swarm simulation).
> >> >
> >> > Can I prevent this behaviour?
> >> > In my mesaging architecture, each agent connects to 6 sockets, so with
> >> 100
> >> > agents I end up with 600 occupied file descriptors :-(
> >> >
> >> > Alexander
> >> >
> >> >
> >> > _______________________________________________
> >> > zeromq-dev mailing list
> >> > zeromq-dev at lists.zeromq.org
> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>
> >>
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160719/f7ab21db/attachment.htm>