[zeromq-dev] Problems with ZeroMQ 4.1.2 and multicast transports on Windows

[Andrew Novikov]

Hello everyone,

I am trying to implement a rather simple file distributing application
using ZeroMQ and its multicast transports (mainly EPGM, but it is
desired to have a working NORM transport too in the end). The
application is similar in purpose to FileMQ but uses its own protocol
and it is imperative that a real multicast transport be used.

The application is designed to work in modern Windows operating
systems. The ultimate goal is to use available network bandwidth
efficiently to distribute copies of files to a relatively large number
(~200) of clients over a LAN.

While implementing the application I came across several problems though.

The main problem is that at the moment it does not seem possible to
make ZeroMQ give away multicast messages at speeds faster than circa
10 megabits per second over 1 GigE. I have written a simple
application (actually two simple programs – zmqsend and zmqreceive) to
test that, it can be found here:
https://github.com/briskycat/ZMQLoadTest It is written in C++11 and
can be compiled with VS2013 or Clang (GCC should do too).

If ZMQ_RATE of the multicast socket is lower than about 8 megabits per
second, packets are sent to the network at the specified rate. But if
the rate is set higher than that number, messages begin to get queued
and newer transmitted faster than about 9-10 megabits per second. In
fact, it seems that congestions start to appear and the average data
transmission rate drops even lower than 8 megabits per second.
Moreover, if I set the number of I/O threads in the ZMQ context to a
value higher than 1, strange transmission gaps begin to appear: the
zmqsend may stop transmitting data for several dozen seconds and then
suddenly resume only to stop transmitting soon (messages continue to
be generated and queued during the gaps).

The test rig consists of two fairly modern PCs with a handful of RAM
and multicore CPUs running Windows 7. The PCs have gigabit ethernet
adapters and are connected via a 48 Gbps switch:
https://h10057.www1.hp.com/ecomcat/hpcatalog/specs/provisioner/05/J9028B.htm

The ZeroMQ used is the latest stable release 4.1.2 built together with
the latest official OpenPGM library (binary distribution built with
VS2013). Everything is statically linked to zmqsend and zmqreceive.

The main question is: can I expect at least, say, 100Mbps data rates
with that setup? And if I can, what is the cause of the problems I am
experiencing?


Kind regards,
Andrey Novikov