[zeromq-dev] inproc queue performance

Pieter Hintjens pieterh at gmail.com
Mon Feb 4 07:47:03 CET 2013


Good to hear!
On Feb 3, 2013 8:05 PM, "Erik Fears" <strtok at strtok.net> wrote:

> I actually replaced it with a PUSH/PULL after Pieter's response and it
> appears to be significantly faster now. You are correct that I do not need
> the reply from the worker.
>
> Thanks!
> Erik
>
>
> On Sat, Feb 2, 2013 at 1:14 AM, Jean-François Smigielski <
> jf.smigielski at gmail.com> wrote:
>
>> Hello Erik and folks,
>>
>> Have you tried spawning multiple acceptor threads ?
>>
>> I am a ZeroMQ newbie and I might be wrong but..
>>
>>    - with only one acceptor, with need an intermediate device? you could
>>    connect workers sockets to the socket bond in the acceptor.
>>    - have you tried replacing the REQ/REP pattern by a PUSH/PULL
>>    pattern? Do you really need the pointer to be returned to the acceptor
>>    thread? I feel you could parallelize better your jobs in the workers,
>>    without waiting for a reply for each connection. What I read in your
>>    exemple is that you are benchmark accept() more than ZMQ.
>>
>>
>> --
>> Jean-François SMIGIELSKI
>> +33 (0) 625 135 563
>>
>>
>> 2013/2/2 Erik Fears <strtok at strtok.net>
>>
>>>  I'm using zeromq 3.2.2, inproc sockets, and the ZMQ_QUEUE device to
>>> dispatch incoming
>>> TCP connections from an acceptor thread to 8 worker threads.
>>>
>>> I'm finding that I can only send about 8500 messages/second over the
>>> queue to worker threads.
>>>
>>> I've tried reducing the amount of work done outside zmq as much as
>>> possible to isolate the performance
>>> problem.
>>>
>>> Any ideas? Is the queue code expected to be this slow?
>>>
>>> Here's the setup:
>>>
>>> 1. There's a pthread that starts that has the purpose of being the
>>> queue. Here's how it's started:
>>>
>>>     void *frontend_socket = zmq_socket(zmq_context(), ZMQ_ROUTER);
>>>     void *backend_socket = zmq_socket(zmq_context(), ZMQ_DEALER);
>>>     zmq_bind(frontend_socket, "inproc://listener-queue-frontend");
>>>     zmq_bind(backend_socket,  "inproc://listener-queue-backend");
>>>     boost::thread dispatch_thread(boost::bind(&dispatch_thread_main,
>>> frontend_socket, backend_socket));
>>>
>>>
>>> void
>>> dispatch_thread_main(void *frontend_socket, void *backend_socket)
>>> {
>>>     zmq_device(ZMQ_QUEUE, frontend_socket, backend_socket);
>>>     zmq_close(frontend_socket);
>>>     zmq_close(backend_socket);
>>> }
>>>
>>>
>>> 2. There's a second thread that's purpose is to receive new connections
>>> (accept()), and then
>>> send them to the queue. Here's what the code looks like:
>>>
>>>     void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REQ);
>>>
>>>     if (zmq_connect(dispatch_socket, "inproc://listener-queue-frontend")
>>> != 0) {
>>>             // handle error
>>>     }
>>>
>>>     while (int new_client_fd = accept(listen_fd, (sockaddr*)
>>> &client_addr,
>>>                                      &client_addr_len))
>>>     {
>>>
>>>         //Sends to dispatcher and should immediately unblock
>>>         zmq_send_ptr<int>(dispatch_socket, &new_client_fd);
>>>
>>>         //Block zmq_recv_ptr will block until the reply
>>>         int *fd = zmq_recv_ptr<int>(dispatch_socket);
>>>         assert(fd == &new_client_fd);
>>>     }
>>>
>>>
>>> 3. N number of works (e.g. 8) are started that execute the following
>>> code. All I'm doing right now
>>>     is calling close() on the accepted socket in the worker thread.
>>>
>>>       void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REP);
>>>
>>>         if (zmq_connect(dispatch_socket,
>>> "inproc://listener-queue-backend") != 0) {
>>>           //handle error
>>>         }
>>>
>>>         int *client_fd = NULL;
>>>        while ((client_fd = zmq_recv_ptr<int>(dispatch_socket, 0)) !=
>>> nullptr) {
>>>                 close(*client_fd);
>>>                 zmq_send_ptr<int>(dispatch_socket.get(), client_fd);
>>>         }
>>>
>>>
>>> 4. zmq_recv_ptr/send_ptr are just some helper functions designed to
>>> implement 0copy message
>>>    passing of pointers. They use zmq_msg_recv/send, and
>>> zmq_msg_init_data. I don't think these
>>>    are the source of any performance issues.
>>>
>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130204/89a32fe1/attachment.html>


More information about the zeromq-dev mailing list