[zeromq-dev] zmq_poll performance question

[Arnaud Kapp]

Hello,

I recently added support for POLLPRI flag.
It looks like it's not handled in your patch and that it needs custom
support. Since there is no test related to this flags you wouldn't
notice.

I can give it a look if you want.

On Sat, Nov 22, 2014 at 2:16 PM, Pieter Hintjens <ph at imatix.com> wrote:
> I suggest you send the patch to libzmq master, and ensure all test
> cases pass. Then we can get this into the next version.
>
> On Fri, Nov 21, 2014 at 2:50 PM, Francis Le Bourse
> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>> Hi,
>>
>> On 11/6/2014 3:18 PM, Pieter Hintjens wrote:
>>>
>>> Oh, ok. Sounds like you have a good candidate for some before/after
>>> measurement and optimization. Are you going to try to make a patch for
>>> this?
>>
>> I have a patch candidate for this optimization, the performance improvement
>> is very good and it doesn't seem to introduce any new instability.
>> What is modified:
>>     - zmq_poll(), there is only one poll() now,
>>     - and epoll() from epoll.cpp
>> Other calls to poll() and select() are left unmodified.
>>
>> I woulld be happy to have any feedback.
>>
>>
>> Cheers,
>> Francis
>>>
>>>
>>> On Thu, Nov 6, 2014 at 2:09 PM, Francis Le Bourse
>>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>>>
>>>> On 11/6/2014 11:47 AM, Pieter Hintjens wrote:
>>>>>
>>>>> A simple optimization is, when you are polling sockets for input, to
>>>>> continue reading from an active socket using a non-blocking read. So
>>>>> you process all waiting messages on a socket and then only switch back
>>>>> to poll when needed.
>>>>
>>>> Thank you for you quick reply.
>>>>
>>>> Yes, but the question was more about the zmq_poll() internals.
>>>> For 600+ file descriptors, zmq_poll() calls poll() a huge number of times
>>>> for only a few that will trigger a POLLIN and the relevant information is
>>>> already known / present in the pollfds array. The performance hit is
>>>> there.
>>>>
>>>> Cheers,
>>>> Francis
>>>>
>>>>
>>>>
>>>>> On Thu, Nov 6, 2014 at 11:28 AM, Francis Le Bourse
>>>>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am looking at a performance issue in zmq, when the number of zsockets
>>>>>> /
>>>>>> file descriptors becomes large.
>>>>>> The relevant calls are:
>>>>>>       poll+0x57
>>>>>>       zmq_poll+0x2e3
>>>>>>       zloop_start+0x1e8
>>>>>>       main+0xb40
>>>>>>       __libc_start_main+0xfd
>>>>>> immediately followed by a loop of
>>>>>>       poll+0x57
>>>>>>       zmq::signaler_t::wait(int)+0x33
>>>>>>       zmq::mailbox_t::recv(zmq::command_t*, int)+0x78
>>>>>>       zmq::socket_base_t::process_commands(int, bool)+0xbe
>>>>>>       zmq::socket_base_t::getsockopt(int, void*, unsigned long*)+0x135
>>>>>>       zmq_getsockopt+0x75
>>>>>>       zmq_poll+0x3da
>>>>>>       zloop_start+0x1e8
>>>>>>       main+0xb40
>>>>>>       __libc_start_main+0xfd
>>>>>>
>>>>>> The code in the loop is executed once per file descriptor in the
>>>>>> initial
>>>>>> pollarray, redoing a poll() system call each time.
>>>>>> Is there a reason to proceed that way ?
>>>>>> Would be possible to reuse the results of the first poll() in order to
>>>>>> bypass the second set of system calls ?
>>>>>>
>>>>>> Cheers,
>>>>>> Francis
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> zeromq-dev mailing list
>>>>>> zeromq-dev at lists.zeromq.org
>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>
>>>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev



-- 
Kapp Arnaud - Xaqq