[zeromq-dev] Distributed application with REQ REPLY sockets

[Patrick]

It’s a 10Gbit network.  

Each of the 50 interfaces (one per node) is sending 150Mbytes while it is receiving 150Mbytes of data.

Patrick

>> On Aug 13, 2018, at 11:31 AM, Justin Azoff <justin.azoff at gmail.com> wrote:
>> 
>> On Mon, Aug 13, 2018 at 2:07 AM Patrick Reedy <preedy73 at gmail.com> wrote:
>> 
>> I’m struggling with a pipeline stage that needs to complete quickly, in less than a second, but sometimes is taking 5 or 6 seconds.
>> 
>> In this pipeline stage each of the 50 nodes have to send their to piece of data to all of the 100 main workers - so 50 nodes each sending 100 pieces of data with each piece about 1.5M bytes. The main workers need to collect all of the pieces of data before the pipeline can advance to the next stage.
> 
> What kind of network interfaces do you have?  When this stage runs how
> much data total does each physical interface send/retrieve?  Could you
> be saturating the network?
> 
> 
> 
> -- 
> - Justin
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev