[zeromq-dev] architecture performance issues

[Andrew Hume]

i think i need to rework some of my architecture because its not scaling
like i need it to.

what is have is 80 processes spread evenly across 8 servers producing output
for a configurable number (say 30) of receiver processes. each message produced
has a marker on it, and all messages with the same tag must go to the
same receiver process (this is to do deduping).

i wanted to avoid each receiver opening up 80 sockets and vice versa,
so i set up (essentially) a forwarder device, one per server,
so that the 80 processes now just open 8 sockets each,
an dthe receiver processes are now anonymous and just connect to
the forwarder on their node.

this doesn't scale becaus ethe forwarder gets too loaded.
right now, i use pub/sub, but would rather use push/pull.

any sggestions on how else i might do it?

------------------
Andrew Hume  (best -> Telework) +1 623-551-2845
andrew at research.att.com  (Work) +1 973-236-2014
AT&T Labs - Research; member of USENIX and LOPSA




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110520/385a38d2/attachment.htm>