[zeromq-dev] design question

[James Gatannah]

On Sat, Oct 27, 2018 at 5:04 SIMON BABY <simonkbaby at gmail.com> wrote:

> Hi,
>
> I have a design requirement. Can anyone  please help.
>
> I have multiple connection from client to server and my client sends a very
> large file, say 4GB size to server. The file is send by splitting into
> chunks and sending each chunk to the different channels (sockets created
> through multiple interfaces) to achieve load balancing. The same client can
> connect to more than 1 server. Currently maximum 16 server.  Attached is a
> sample connection diagram from client to server. Can you please help if I
> can design this with RabbitMQ/AMQP ?

There are lots of ways to skin this cat. You'll have to balance the
trade-offs in a way that makes the most sense for you.

I personally would never use AMQP for moving around large amounts of bytes.

If I had to do this with AMQP, I'd probably send a message that
contains a URL for each chunk, have the client download them using
HTTP, then stick a load-balancer in front of the file servers.

http://zguide.zeromq.org/hx:chapter7#toc22 is a good starting point
for reliable file transfer using zeromq, though it doesn't get into
any sort of load balancing scenarios (I don't think...it's been
several years since I actually read it).

Good luck,
James


> Rgds
> Simon