[zeromq-dev] Can I prevent zeromq from occupying file descriptors?

[Alexander Poddey]

Thanks Doron, Tony and Michal.
I made a test in the meantime and found what Michal said: every connection 
needs a fd.
I followed Tony's advice: the admin can allow a higher soft/hard limit on a 
per user basis; this then seems to be the way to go. One needs to convince 
the cluster admins to do that.

An accompanying measure seems to be to use the new resource property 
(http://zmtp.org/page:read-the-docs#toc18), which seems to allow the 
combination of several sockets (like dealer, pub services, which I have 6 in 
my implementation) into one e.g. TCP endpoint. 
I guess using this technique I would need one file dscriptor where my 
current implementation uses 6...


Thanks for your support!

Alexander



Doron Somech wrote:

> I think IPC also use FD per connection but not sure. PGM is one FD per
> sender/receiver, I think.
> 
> However,  I don't think you should worry about that,  OS is there to
> handle
> ten of thousands of FD,  just configure correctly.  Also maybe linux
> server and linux desktop have different settings.
> 
> On Jul 19, 2016 20:49, "Alexander Poddey" <alexander.poddey at gmx.net>
> wrote:
> 
>>
>> Ah, ok, now I understand.
>> I guess using IPC should solve the problem for processes on the same
>> cluster-node. If so, I could use a tcp bride between nodes, and IPC
>> locally.
>>
>> How about PGM?
>>
>> Alexander
>>
>>
>> Doron Somech wrote:
>>
>> > It is not zeromq, its TCP, when you have 10K open connections you are
>> > using 10K file descriptors, there is no wonder here, that how TCP
>> > works. HAProxy and nginx for example do the same, or any webserver, if
>> > you want to support 10K concurrent connections you must increase the
>> > maximum sockets per application.
>> >
>> > Take a look a the accept socket docs:
>> >
>> > http://man7.org/linux/man-pages/man2/accept.2.html
>> >
>> > On Tue, Jul 19, 2016 at 7:12 PM, Alexander Poddey
>> > <alexander.poddey at gmx.net> wrote:
>> >
>> >> Hmm,
>> >>
>> >> on linux, the number of file descriptors per process is limited
>> >> (typically to 1024). The total limit on my machine (kernel limit is
>> >> ~734 000 opened files).
>> >>
>> >> This means a process can accept only 1024 connections?!?
>> >>
>> >> I can tweak the limit as super user of that machine; this however is
>> >> no good
>> >> solution.
>> >> Think of my agent simulation running on a server where I'm not root.
>> Then
>> >> each process can accept only 1024 connections which is way to few...
>> >>
>> >> I wonder why zmq needs a file descriptor per connection...
>> >>
>> >> Alexander
>> >>
>> >>
>> >>
>> >> Doron Somech wrote:
>> >>
>> >> > I'm  not sure I  understand,  each file descriptor is a socket. 
>> >> > When zeromq accept tcp socket another file descriptor is created. OS
>> >> > should be able to manage thousands and tens of thousand open
>> >> > descriptor.
>> >> >
>> >> > On Jul 19, 2016 6:56 PM, "Alexander Poddey"
>> >> > <alexander.poddey at gmx.net
>> >
>> >> > wrote:
>> >> >
>> >> >>
>> >> >> Essentially, the question is:
>> >> >> why does ZMQ consume file descriptors at all?
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> > Hi,
>> >> >> >
>> >> >> > I have a setup in C++ on linux where a master process is setting
>> >> >> > up a ZMQ_ROUTER and then forks many child processes, which then
>> connect
>> >> >> > to
>> >> >> that
>> >> >> > router (tcp protocol).
>> >> >> > Whenever a child zmq_connect's to the master, a file descriptor
>> >> >> > is occupied (and assigned to the master process). They get
>> >> >> > free'ed
>> fine
>> >> >> > when the connection is close.
>> >> >> >
>> >> >> > This however limits the number of interacting processes to the
>> >> >> > number of allowed file descriptors (per process). For me (linux),
>> >> >> > this currently is 1024, which is way to small for my intended use
>> >> >> > (multi-agent / swarm simulation).
>> >> >> >
>> >> >> > Can I prevent this behaviour?
>> >> >> > In my mesaging architecture, each agent connects to 6 sockets, so
>> >> >> > with
>> >> >> 100
>> >> >> > agents I end up with 600 occupied file descriptors :-(
>> >> >> >
>> >> >> > Alexander
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > zeromq-dev mailing list
>> >> >> > zeromq-dev at lists.zeromq.org
>> >> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> zeromq-dev mailing list
>> >> >> zeromq-dev at lists.zeromq.org
>> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >>
>> >>
>> >> _______________________________________________
>> >> zeromq-dev mailing list
>> >> zeromq-dev at lists.zeromq.org
>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev