[zeromq-dev] patch: handle idle connections
Martin Sustrik
sustrik at fastmq.com
Tue Mar 24 10:13:57 CET 2009
Dhammika,
Wow! You are the first person out there who ever dared to touch 0MQ core
code :)
And there's actually a token in the bug tracker to do this kind of thing:
http://jira.fastmq.org/browse/ZMQ-130
Your patch goes it the right direction by eliminating the need to browse
over large number of pipes that are idle anyway, however, on the other
hand it introduces a new problem of it's own:
- typedef std::vector <pipe_t*> pipes_t;
+ typedef std::list <pipe_t*> pipes_t;
This means that every time pipe is moved from one list to another one
deallocation (free) and one allocation (malloc) is done - list stores
its elements in dynamically allocated slots.
This may become a problem in scenarios where lot of pipes are swiftly
moving between active and idle states (mostly reqest/reply or ping-pong
style scenarios).
To actually measure the performance impact you would need a multicore
box where allocations/deallocations are done on different CPU cores. In
that case the overhead is not-negligible even with optimised memory
allocators like tcmalloc.
Thus, to overcome this problem we've optimised 0MQ heavily to allocate
as little as possible. At the current stage there's an
allocation/deallocation pair done once per 256 messages.
With list<> implementation of active/idle lists you have to do 1
allocation per message (in the worst case scenario) rising the total
from 0.003 alloc/dealloc pairs per message to 1.003 alloc/dealloc pairs
per message.
Possible solution would be to implement a simple double-linked list that
would be able to move elements between two instances efficiently (just
rewiring the pointers in the list rather than allocating/deallocating
new element (see attached diagram):
my_list_t <pipe_t*> active;
my_list_t <pipe_t*> idle;
// Insert an element to the list.
// Allocation happens here.
active.push_back (pipe);
// Move an element from one list to another.
// No alloc/dealloc happens.
my_list_t <pipe_t*>::iterator it = active.begin ();
active.move_back (it, idle);
This kind of solution would require allocation only when new connection
is established. There will be no additional allocations/deallocations on
the critical path.
Thoughts?
Martin
Dhammika Pathirana wrote:
> Hi,
>
> A patch to improve handling idle connections.
> I did bit of testing on linux.
>
> Dhammika
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: list.png
Type: image/png
Size: 13537 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20090324/82aff135/attachment.png>
More information about the zeromq-dev
mailing list