[zeromq-dev] Issues with large numbers of clients

Will Moss wmoss at bu.mp
Sun Jan 27 02:46:26 CET 2013


Pieter,

I'm using the same client identity to demonstrate the point that the ROUTER
socket cannot possibly be removing the routing information when the client
disconnects, since when it reconnects later, the data is still buffered and
ready to be sent out.

I don't know how I would make a test case that demonstrated the internal
workings of the ROUTER socket when there are no methods exposed to show the
size of that table or anything else like that. If you just had a process
that connected and disconnected from another process like the server
process I wrote, I'm sure you would see the memory increase (as well as the
cpu load, as it has to scan a larger and larger table) and not drop back
down.

Will


On Sat, Jan 26, 2013 at 6:40 AM, Pieter Hintjens <ph at imatix.com> wrote:

> You are connecting two peers using the same identity. This is illegal
> and will result in unspecified behavior.
>
> Unless you have a strong reason for it, don't set socket identities at all.
>
> Can you make a test case that doesn't set socket identities?
>
> -Pieter
>
> On Sat, Jan 26, 2013 at 12:02 AM, Will Moss <wmoss at bu.mp> wrote:
> > Hi Peter,
> >
> > I have looked through the internals a bit, but certainly don't understand
> > them entirely. That said, I don't think it's possible for it to clean up
> on
> > disconnect--if it did it would not be providing the abstraction
> surrounding
> > client reconnections that it promises. I made a little test program to
> prove
> > this to myself: https://gist.github.com/662192fac6fb9d77fc2b
> >
> > Let me know if I can provide any more assistance with the epoll bug. I'd
> > love to get it fixed because the behaviour when this occurs is to have
> the
> > process cpu spike up to 100% and start processing substantially fewer
> > requests that it should (resulting in us having to restart the process
> and
> > lose data in flight).
> >
> > Thanks,
> > Will
> >
> >
> > On Wed, Jan 23, 2013 at 12:16 PM, Pieter Hintjens <ph at imatix.com> wrote:
> >>
> >> On Wed, Jan 23, 2013 at 8:01 PM, Will Moss <wmoss at bu.mp> wrote:
> >>
> >> > I can certainly make a test case for the socket leakage, but as I said
> >> > before, I thought this was a known issue, so I'm a little confused.
> >>
> >> The issue the Guide talks about is application code that tracks peers.
> >> I've not tested, and it's not documented, how the ROUTER socket
> >> manages its internal tables. I'd expect the router socket to clean up
> >> its internal tables when clients disconnect. So if it's not doing
> >> this, we need a test case.
> >>
> >> > I doubt I'm going to be able to come up with a test case that can get
> a
> >> > socket into EPOLLERR or EPOLLHUP, but I think it's reasonably easy to
> >> > see
> >> > how this can happen from the code.
> >>
> >> That may be enough to fix the problem.
> >>
> >> -Pieter
> >>
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130126/1603a781/attachment.htm>


More information about the zeromq-dev mailing list