[zeromq-dev] Handling of fd-limit in Java

Johan Ström johan at stromnet.se
Thu Nov 10 16:55:42 CET 2011


I'm doing some work with ZMQ 2.1.0, and the latest Java binding. I got one thread with an inproc ROUTER socket, and then several threads doing REQ to this ROUTER.
While testing this out and benchmarking a bit, I noticed that creating too many REQ-sockets left me out of FDs (even if I closed them sequentially inbetween, but I assume that has to do with the way ZMQ does closes async.).

Now, going out of FDs is expected, but the problem is, whenever I hit the FD limit, I hit an assert:

    [junit] Too many open files
    [junit] rc == 0 (signaler.cpp:330)

And an assert like this yields a total termination of the JVM it seems. Not so nice in an app server context for example.

Any thoughts on alternative ways to handle this kind of situations such as returning null, letting the caller decide wether to assert or otherwise handle it gracefully?

Easy way to reproduce:

                for(int i  = 0; i < 20000;i++) {
                        ZMQ.Socket s = ctx.socket(ZMQ.REQ);

Also, I've seen some weird crashes related in the same tests, my app opens a REQ socket, connects to the ROUTER, waits for reply, and then closes it. If i put a heavy load on these sockets, I eventually crash with the following error:

Device or resource busy (mutex.hpp:91)

From the code it looks like it's pthread_mutex_destroy which fails. 
I'll probably go for keeping the socket in a ThreadLocal for now, avoiding the re-creation of the socket, but just wanted to report it anyway.

Please reply directly to me, I only get the digests, will be hard to do followup answers with only that.


More information about the zeromq-dev mailing list