[zeromq-dev] architecture performance issues

[Andrew Hume]

i think its just the sheer volume.
the work done per message is pretty minimal:
it basically checks a header field to see if it should be dropped
and then (if not) simply resend.

its not a problem exactly now; they run at an average of 30% cpu.
but i am only running on 10% of the full scale input.

i guess i can eliminate the forwarders; its one less process to worrry about.
adressing is not a concern; the receivers are anonymous now, and they can
remain so, anchoring to the 80 processes rather than just to teh one device.

i guess we have nothing to fear but fear itself.

thanks, pieter.

		andrew

On May 20, 2011, at 2:37 AM, Pieter Hintjens wrote:

> On Fri, May 20, 2011 at 11:18 AM, Andrew Hume <andrew at research.att.com> wrote:
> 
>> i think i need to rework some of my architecture because its not scaling
>> like i need it to.
> 
> Could you explain why the forwarders are getting overloaded? What's
> your message volume, message size, and work per message in the
> forwarder?
> 
> Normally, you could expect devices to become bottlenecks, but that's
> balanced against them doing much less work per message than endpoints.
> And a device will essentially be CPU bound. So if a device starts to
> do too much work per message, it'll degrade. Or, if the overall
> volumes combine to more than the device can handle, it'll degrade.
> 
> Options are therefore to reduce the number of messages, to reduce the
> work done per message, in devices, and eventually to remove devices
> altogether.
> 
> The pattern which perhaps works best for such a network is a single
> device for coordination and registration, but all traffic flowing peer
> to peer. 80 sockets per node is not an issue, you can open thousands
> if needed. The issue is usually to address these, and you can solve
> that with a broker / name server device.
> 
> It's not been covered in the Guide yet so whatever you come up with,
> please do share, it'll be useful to many people.
> 
> -Pieter
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev


------------------
Andrew Hume  (best -> Telework) +1 623-551-2845
andrew at research.att.com  (Work) +1 973-236-2014
AT&T Labs - Research; member of USENIX and LOPSA




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110520/ad711943/attachment.htm>