[zeromq-dev] architecture performance issues
andrew at research.att.com
Fri May 20 12:17:58 CEST 2011
i think its just the sheer volume.
the work done per message is pretty minimal:
it basically checks a header field to see if it should be dropped
and then (if not) simply resend.
its not a problem exactly now; they run at an average of 30% cpu.
but i am only running on 10% of the full scale input.
i guess i can eliminate the forwarders; its one less process to worrry about.
adressing is not a concern; the receivers are anonymous now, and they can
remain so, anchoring to the 80 processes rather than just to teh one device.
i guess we have nothing to fear but fear itself.
On May 20, 2011, at 2:37 AM, Pieter Hintjens wrote:
> On Fri, May 20, 2011 at 11:18 AM, Andrew Hume <andrew at research.att.com> wrote:
>> i think i need to rework some of my architecture because its not scaling
>> like i need it to.
> Could you explain why the forwarders are getting overloaded? What's
> your message volume, message size, and work per message in the
> Normally, you could expect devices to become bottlenecks, but that's
> balanced against them doing much less work per message than endpoints.
> And a device will essentially be CPU bound. So if a device starts to
> do too much work per message, it'll degrade. Or, if the overall
> volumes combine to more than the device can handle, it'll degrade.
> Options are therefore to reduce the number of messages, to reduce the
> work done per message, in devices, and eventually to remove devices
> The pattern which perhaps works best for such a network is a single
> device for coordination and registration, but all traffic flowing peer
> to peer. 80 sockets per node is not an issue, you can open thousands
> if needed. The issue is usually to address these, and you can solve
> that with a broker / name server device.
> It's not been covered in the Guide yet so whatever you come up with,
> please do share, it'll be useful to many people.
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
Andrew Hume (best -> Telework) +1 623-551-2845
andrew at research.att.com (Work) +1 973-236-2014
AT&T Labs - Research; member of USENIX and LOPSA
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the zeromq-dev