[zeromq-dev] How to increase Max Socket count per context?

Lookis lookisliu at gmail.com
Thu Jul 25 08:22:32 CEST 2013


Hi, all 
i found zeromq very useful , and there is a massive usage in my company. recently i start a new project with zeromq, but get some problem. i intend to to use zeromq as a server that serve millions of connections at the same time, you can imagine it's like a push server, i need to keep all client's connection by socket like tcp keepalive or something. first of all, i designed a performance test for the server and client side. i use ROUTER socket for server, everything for server is OK. 

when i try to wrote client side, i plan to use 1 thread/process(for minimal CPU/memory cost), and this only thread polling multi socket's io. i met exception "Too many open files" when creating over 1024 sockets in one context(after setting ulimit -SHm 1024000, and append "* - nofile 1024000" in /etc/security/limits.conf ).so i have to create multi-context in one thread, when i try to create 40+ contexts, there is a crash of jvm :(

how can i create many connections with minimal resource(thread/process/memory) to get through the test? i am trying to serve 100millions/server, is this possible?

my env is: jdk 7 + zeromq 3.2.3 + jzmq v2.2.0 + centos 6.4
and compiled zeromq with argument "--with-poller=epoll"

here are some code of my client side:


    private final int SOCKET_PER_CONTEXT = 1024; 

    private final int CONTEXT_PER_CLIENT = 40; 

    private Poller poller = null; 

    private Map<Socket, ConcurrentLinkedQueue<byte[]>> sockets = null; 

    public Client(int clientId) { 
        poller = new ZMQ.Poller(SOCKET_PER_CONTEXT * CONTEXT_PER_CLIENT);
        sockets = new HashMap<>(SOCKET_PER_CONTEXT * CONTEXT_PER_CLIENT);
        for (int i = 0; i < CONTEXT_PER_CLIENT; i++) {
            Context context = ZMQ.context(1);
            for (int j = 0; j < SOCKET_PER_CONTEXT; j++) {
                ZMQ.Socket frontend = context.socket(ZMQ.DEALER);
                frontend.setIdentity(("" + clientId + "|" + i).getBytes());
                frontend.connect("tcp://127.0.0.1:5257");
                sockets.put(frontend, new ConcurrentLinkedQueue<byte[]>());
                poller.register(frontend);
            }
        }
    }



-- 
Lookis
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130725/4e4a1eac/attachment.htm>


More information about the zeromq-dev mailing list