[zeromq-dev] inproc queue performance
Erik Fears
strtok at strtok.net
Sun Feb 3 20:05:01 CET 2013
I actually replaced it with a PUSH/PULL after Pieter's response and it
appears to be significantly faster now. You are correct that I do not need
the reply from the worker.
Thanks!
Erik
On Sat, Feb 2, 2013 at 1:14 AM, Jean-François Smigielski <
jf.smigielski at gmail.com> wrote:
> Hello Erik and folks,
>
> Have you tried spawning multiple acceptor threads ?
>
> I am a ZeroMQ newbie and I might be wrong but..
>
> - with only one acceptor, with need an intermediate device? you could
> connect workers sockets to the socket bond in the acceptor.
> - have you tried replacing the REQ/REP pattern by a PUSH/PULL pattern?
> Do you really need the pointer to be returned to the acceptor thread? I
> feel you could parallelize better your jobs in the workers, without waiting
> for a reply for each connection. What I read in your exemple is that you
> are benchmark accept() more than ZMQ.
>
>
> --
> Jean-François SMIGIELSKI
> +33 (0) 625 135 563
>
>
> 2013/2/2 Erik Fears <strtok at strtok.net>
>
>> I'm using zeromq 3.2.2, inproc sockets, and the ZMQ_QUEUE device to
>> dispatch incoming
>> TCP connections from an acceptor thread to 8 worker threads.
>>
>> I'm finding that I can only send about 8500 messages/second over the
>> queue to worker threads.
>>
>> I've tried reducing the amount of work done outside zmq as much as
>> possible to isolate the performance
>> problem.
>>
>> Any ideas? Is the queue code expected to be this slow?
>>
>> Here's the setup:
>>
>> 1. There's a pthread that starts that has the purpose of being the queue.
>> Here's how it's started:
>>
>> void *frontend_socket = zmq_socket(zmq_context(), ZMQ_ROUTER);
>> void *backend_socket = zmq_socket(zmq_context(), ZMQ_DEALER);
>> zmq_bind(frontend_socket, "inproc://listener-queue-frontend");
>> zmq_bind(backend_socket, "inproc://listener-queue-backend");
>> boost::thread dispatch_thread(boost::bind(&dispatch_thread_main,
>> frontend_socket, backend_socket));
>>
>>
>> void
>> dispatch_thread_main(void *frontend_socket, void *backend_socket)
>> {
>> zmq_device(ZMQ_QUEUE, frontend_socket, backend_socket);
>> zmq_close(frontend_socket);
>> zmq_close(backend_socket);
>> }
>>
>>
>> 2. There's a second thread that's purpose is to receive new connections
>> (accept()), and then
>> send them to the queue. Here's what the code looks like:
>>
>> void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REQ);
>>
>> if (zmq_connect(dispatch_socket, "inproc://listener-queue-frontend")
>> != 0) {
>> // handle error
>> }
>>
>> while (int new_client_fd = accept(listen_fd, (sockaddr*) &client_addr,
>> &client_addr_len))
>> {
>>
>> //Sends to dispatcher and should immediately unblock
>> zmq_send_ptr<int>(dispatch_socket, &new_client_fd);
>>
>> //Block zmq_recv_ptr will block until the reply
>> int *fd = zmq_recv_ptr<int>(dispatch_socket);
>> assert(fd == &new_client_fd);
>> }
>>
>>
>> 3. N number of works (e.g. 8) are started that execute the following
>> code. All I'm doing right now
>> is calling close() on the accepted socket in the worker thread.
>>
>> void *dispatch_socket = zmq_socket(zmq_context(), ZMQ_REP);
>>
>> if (zmq_connect(dispatch_socket,
>> "inproc://listener-queue-backend") != 0) {
>> //handle error
>> }
>>
>> int *client_fd = NULL;
>> while ((client_fd = zmq_recv_ptr<int>(dispatch_socket, 0)) !=
>> nullptr) {
>> close(*client_fd);
>> zmq_send_ptr<int>(dispatch_socket.get(), client_fd);
>> }
>>
>>
>> 4. zmq_recv_ptr/send_ptr are just some helper functions designed to
>> implement 0copy message
>> passing of pointers. They use zmq_msg_recv/send, and
>> zmq_msg_init_data. I don't think these
>> are the source of any performance issues.
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130203/23d6f2c9/attachment.htm>
More information about the zeromq-dev
mailing list