[zeromq-dev] Assertion failed: fds.size() <= FD_SETSIZE in select.cpp:67

Troy Howard thoward37 at gmail.com
Fri Jan 14 00:38:16 CET 2011


I ran into this same issue on my Windows box install today. I was in
the process of putting together a new system for handling queries on a
web site. I had a web front end (linux), a queue device broker
(windows), and numerous workers (windows) in the backend processing
queries.

I am using pyzmq in python for all three pieces, and using the queue
device for the broker. I'm using 0mq version 2.0.10.

During initial testing the queue/broker was crashing with the same
'FD_SETSIZE; error described here. This was happening after hitting it
with only minor load (a few hundred requests in a tight loop in a
testing app). After reading this thread I immediately assumed it was
the Windows environment causing my issue. Our backend is Windows
(because some of the queries require windows specific stuff). I tried
moving the broker to one of the Linux boxes where the web front end
lives.

I encountered a similar issue ('Too many open files sock != -1
(tcp_listener.cpp:316)' ).

Then I looked in my worker code... There was a bug there that was
responsible for this. There's a primary loop where it connects to the
broker, grabs the message, processes, sends results, rinse repeat.
Unfortunately the call to connect the socket was INSIDE the loop
instead of outside where it should have been.

I moved the broker back to Windows after having fixed the worker
connection code, and the problem disappeared. I hit it with
increasingly more load, from 100 to 100,000 queries in a tight loop
from a single client and did not encounter any problems with the
broker. I tried 10,000 requests each from two concurrent client
connections, also no problem. I imagine I won't see this problem
again, unless I open my broker to a lot more endpoints.

So, I suggest, before modifying the FD_SETSIZE constant, that you
inspect your client/worker code for correct use and disposal of socket
connections. This may or may not help in your case.

Thanks,
Troy



More information about the zeromq-dev mailing list