[zeromq-dev] zmq_poll performance question

Arnaud Kapp kapp.arno at gmail.com
Mon Nov 24 20:08:45 CET 2014


> Currently, the patch is written for 3.2.4. I'll wait to put it in libzmq master

Oh okay. This is the commit that added the flag:
https://github.com/zeromq/libzmq/commit/779c37abc433cb6595ddeedaf86b280317656bdd

libzmq was 4.1 at the time I believe.

I'll probably look at it this week-end then :)

On Mon, Nov 24, 2014 at 12:10 PM, Francis Le Bourse
<zno-reply-francis.lebourse at sfr-sh.fr> wrote:
> Hi,
> On 11/24/2014 11:35 AM, Arnaud Kapp wrote:
>>
>> Hello,
>>
>> I recently added support for POLLPRI flag.
>> It looks like it's not handled in your patch
>
> No, it isn't handled. In which version do you have added this flag ?
> Currently, the patch is written for 3.2.4. I'll wait to put it in libzmq
> master.
>>
>>   and that it needs custom
>> support. Since there is no test related to this flags you wouldn't
>> notice.
>>
>> I can give it a look if you want.
>
> That would be nice.
>
> Cheers,
> Francis
>
>
>>
>> On Sat, Nov 22, 2014 at 2:16 PM, Pieter Hintjens <ph at imatix.com> wrote:
>>>
>>> I suggest you send the patch to libzmq master, and ensure all test
>>> cases pass. Then we can get this into the next version.
>>>
>>> On Fri, Nov 21, 2014 at 2:50 PM, Francis Le Bourse
>>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 11/6/2014 3:18 PM, Pieter Hintjens wrote:
>>>>>
>>>>> Oh, ok. Sounds like you have a good candidate for some before/after
>>>>> measurement and optimization. Are you going to try to make a patch for
>>>>> this?
>>>>
>>>> I have a patch candidate for this optimization, the performance
>>>> improvement
>>>> is very good and it doesn't seem to introduce any new instability.
>>>> What is modified:
>>>>      - zmq_poll(), there is only one poll() now,
>>>>      - and epoll() from epoll.cpp
>>>> Other calls to poll() and select() are left unmodified.
>>>>
>>>> I woulld be happy to have any feedback.
>>>>
>>>>
>>>> Cheers,
>>>> Francis
>>>>>
>>>>>
>>>>> On Thu, Nov 6, 2014 at 2:09 PM, Francis Le Bourse
>>>>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>>>>>
>>>>>> On 11/6/2014 11:47 AM, Pieter Hintjens wrote:
>>>>>>>
>>>>>>> A simple optimization is, when you are polling sockets for input, to
>>>>>>> continue reading from an active socket using a non-blocking read. So
>>>>>>> you process all waiting messages on a socket and then only switch
>>>>>>> back
>>>>>>> to poll when needed.
>>>>>>
>>>>>> Thank you for you quick reply.
>>>>>>
>>>>>> Yes, but the question was more about the zmq_poll() internals.
>>>>>> For 600+ file descriptors, zmq_poll() calls poll() a huge number of
>>>>>> times
>>>>>> for only a few that will trigger a POLLIN and the relevant information
>>>>>> is
>>>>>> already known / present in the pollfds array. The performance hit is
>>>>>> there.
>>>>>>
>>>>>> Cheers,
>>>>>> Francis
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Thu, Nov 6, 2014 at 11:28 AM, Francis Le Bourse
>>>>>>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am looking at a performance issue in zmq, when the number of
>>>>>>>> zsockets
>>>>>>>> /
>>>>>>>> file descriptors becomes large.
>>>>>>>> The relevant calls are:
>>>>>>>>        poll+0x57
>>>>>>>>        zmq_poll+0x2e3
>>>>>>>>        zloop_start+0x1e8
>>>>>>>>        main+0xb40
>>>>>>>>        __libc_start_main+0xfd
>>>>>>>> immediately followed by a loop of
>>>>>>>>        poll+0x57
>>>>>>>>        zmq::signaler_t::wait(int)+0x33
>>>>>>>>        zmq::mailbox_t::recv(zmq::command_t*, int)+0x78
>>>>>>>>        zmq::socket_base_t::process_commands(int, bool)+0xbe
>>>>>>>>        zmq::socket_base_t::getsockopt(int, void*, unsigned
>>>>>>>> long*)+0x135
>>>>>>>>        zmq_getsockopt+0x75
>>>>>>>>        zmq_poll+0x3da
>>>>>>>>        zloop_start+0x1e8
>>>>>>>>        main+0xb40
>>>>>>>>        __libc_start_main+0xfd
>>>>>>>>
>>>>>>>> The code in the loop is executed once per file descriptor in the
>>>>>>>> initial
>>>>>>>> pollarray, redoing a poll() system call each time.
>>>>>>>> Is there a reason to proceed that way ?
>>>>>>>> Would be possible to reuse the results of the first poll() in order
>>>>>>>> to
>>>>>>>> bypass the second set of system calls ?
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Francis
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> zeromq-dev mailing list
>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>
>



-- 
Kapp Arnaud - Xaqq



More information about the zeromq-dev mailing list