[zeromq-dev] zmq_poll performance question

[Francis Le Bourse]

Hi,
On 11/24/2014 11:35 AM, Arnaud Kapp wrote:
> Hello,
>
> I recently added support for POLLPRI flag.
> It looks like it's not handled in your patch
No, it isn't handled. In which version do you have added this flag ?
Currently, the patch is written for 3.2.4. I'll wait to put it in libzmq 
master.
>   and that it needs custom
> support. Since there is no test related to this flags you wouldn't
> notice.
>
> I can give it a look if you want.
That would be nice.

Cheers,
Francis

>
> On Sat, Nov 22, 2014 at 2:16 PM, Pieter Hintjens <ph at imatix.com> wrote:
>> I suggest you send the patch to libzmq master, and ensure all test
>> cases pass. Then we can get this into the next version.
>>
>> On Fri, Nov 21, 2014 at 2:50 PM, Francis Le Bourse
>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>> Hi,
>>>
>>> On 11/6/2014 3:18 PM, Pieter Hintjens wrote:
>>>> Oh, ok. Sounds like you have a good candidate for some before/after
>>>> measurement and optimization. Are you going to try to make a patch for
>>>> this?
>>> I have a patch candidate for this optimization, the performance improvement
>>> is very good and it doesn't seem to introduce any new instability.
>>> What is modified:
>>>      - zmq_poll(), there is only one poll() now,
>>>      - and epoll() from epoll.cpp
>>> Other calls to poll() and select() are left unmodified.
>>>
>>> I woulld be happy to have any feedback.
>>>
>>>
>>> Cheers,
>>> Francis
>>>>
>>>> On Thu, Nov 6, 2014 at 2:09 PM, Francis Le Bourse
>>>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>>>> On 11/6/2014 11:47 AM, Pieter Hintjens wrote:
>>>>>> A simple optimization is, when you are polling sockets for input, to
>>>>>> continue reading from an active socket using a non-blocking read. So
>>>>>> you process all waiting messages on a socket and then only switch back
>>>>>> to poll when needed.
>>>>> Thank you for you quick reply.
>>>>>
>>>>> Yes, but the question was more about the zmq_poll() internals.
>>>>> For 600+ file descriptors, zmq_poll() calls poll() a huge number of times
>>>>> for only a few that will trigger a POLLIN and the relevant information is
>>>>> already known / present in the pollfds array. The performance hit is
>>>>> there.
>>>>>
>>>>> Cheers,
>>>>> Francis
>>>>>
>>>>>
>>>>>
>>>>>> On Thu, Nov 6, 2014 at 11:28 AM, Francis Le Bourse
>>>>>> <zno-reply-francis.lebourse at sfr-sh.fr> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am looking at a performance issue in zmq, when the number of zsockets
>>>>>>> /
>>>>>>> file descriptors becomes large.
>>>>>>> The relevant calls are:
>>>>>>>        poll+0x57
>>>>>>>        zmq_poll+0x2e3
>>>>>>>        zloop_start+0x1e8
>>>>>>>        main+0xb40
>>>>>>>        __libc_start_main+0xfd
>>>>>>> immediately followed by a loop of
>>>>>>>        poll+0x57
>>>>>>>        zmq::signaler_t::wait(int)+0x33
>>>>>>>        zmq::mailbox_t::recv(zmq::command_t*, int)+0x78
>>>>>>>        zmq::socket_base_t::process_commands(int, bool)+0xbe
>>>>>>>        zmq::socket_base_t::getsockopt(int, void*, unsigned long*)+0x135
>>>>>>>        zmq_getsockopt+0x75
>>>>>>>        zmq_poll+0x3da
>>>>>>>        zloop_start+0x1e8
>>>>>>>        main+0xb40
>>>>>>>        __libc_start_main+0xfd
>>>>>>>
>>>>>>> The code in the loop is executed once per file descriptor in the
>>>>>>> initial
>>>>>>> pollarray, redoing a poll() system call each time.
>>>>>>> Is there a reason to proceed that way ?
>>>>>>> Would be possible to reuse the results of the first poll() in order to
>>>>>>> bypass the second set of system calls ?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Francis
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> zeromq-dev mailing list
>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>