[zeromq-dev] hitting file descriptor using gevent

[Tim Cooijmans]

Hi,

I'm having problems building an asynchronous application using zeromq: 

My aim is to have a number of front-end servers running gevent. These
front-end servers will receive HTTP-requests. When they receive a request
they will send off a request to do some calculation to a worker-server via
zeromq, then wait for a reply to come back and when it comes back parse some
data from the response in the HTTP-response body. The waiting can take up to
1 second and the servers should be able to process thousands of request per
second so a asynchronous server is needed.
A worker-server has to be able to process multiple requests at once because
there is a lot of waiting to be done when processing. 

I boiled it down to this simulation:
Front-end-server:
def sendmessage(i):
    # Get context
    context = zmq.Context()
    socket = context.socket(zmq.REQ)
    socket.connect("tcp://127.0.0.1:5000")
    
    # Send message
    socket.send(str(i))
    print("Send: " + str(i))
    
    # Wait for confirmation
    socket.recv()
    print("OK: " + str(i))

all = []
for i in range(10):
    thread = gevent.spawn(sendmessage, i)
    all.append(thread) 
gevent.joinall(all)

Worker-server:
context = zmq.Context()
socket = context.socket(zmq.XREP)
socket.bind('tcp://127.0.0.1:5000')

def worker(socket, message):
    print("Received: " + str(message[2]))
    gevent.sleep(20)
    print("Processed: " + str(message[2]))
    socket.send_multipart([message[0], '', "OK"])   

def master(socket):
    while True:
        message = socket.recv_multipart()
        gevent.spawn(worker, socket, message)
        
gevent.spawn(master, socket).join()

Which works fine as long as there are less like 70 clients. Above this the
"frontend-server" is hitting the file descriptor limit. A possible solution
is to push the file descriptor limit up but this doesn't seem like the right
thing to do to me. 
What is the best way to achieve my goals? Should I abandon the REQ-XREP
architecture?

Best Regards,
Tim