[zeromq-dev] Need 100K+ IPC sockets

Pieter Hintjens ph at imatix.com
Tue Feb 25 21:33:18 CET 2014


First, handling large number of clients. The FD_SETSIZE setting Ahmet
mentions is on Windows. For Linux you have to set unlimited file
handles both at system, and at process level.

Your model is basically pub-sub where the group is a prefix (a topic,
if you like). You can try connecting 100K subscribers to one
publisher. That may work. You can try a fan out model with proxies,
each handling one group, each with their set of subcribers.

On Tue, Feb 25, 2014 at 9:23 PM, Ahmet Kakıcı <ahmetkakici at gmail.com> wrote:
> 2 ) libzmq sets FD_SETSIZE to 1000.
> Pieter Hintjens answered that before http://stackoverflow.com/a/13820909
>
>
> On Tue, Feb 25, 2014 at 10:08 PM, Joe Lin <jlin at peaxy.net> wrote:
>>
>> I'm building a server that should support 100K+ concurrent users. A user
>> belongs to one or more groups and the number of groups would be around 1/10
>> of the total users. Therefore there will be about 10K groups. A user needs
>> to be notified by the events that occur in the groups that he/she belong and
>> the events occur at a  frequency 10/sec/group.
>>
>> My idea is to create a 0MQ broker per group. A user will be connected to
>> one or more brokers via 0MQ IPC socket (maybe a ROUTER socket). When an
>> event is sent to a broker the broker then route the event to the connected
>> clients. Let's say that a user usually belong to 5 groups. It means that,
>> for 100K users and 10K groups I will need to bind 10K IPC sockets and will
>> need 500K IPC socket connection in the server.
>>
>>
>> So I have two questions regarding this:
>>
>> 1. Does this sound like a reasonable use case to use 0MQ IPC socket? Any
>> alternative design?
>>
>> 2. Does ZeroMq limit the number of sockets per context? It seems that I
>> get a failure of "too many open files" although my system has ulimit of
>> "unlmited"
>>
>>
>> Thanks for any help.
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



More information about the zeromq-dev mailing list