[zeromq-dev] How zeromq handles slow consumers with PUB/SUB mode

[Martin Sustrik]

Hi Joel,

> I'm planning to write a prototype using zeromq. I see in the 
> documentation that zeromq provides 
> a publisher/subscriber mode. I want to use this mode in my application, 
> but i'm asking how zeromq handles
> slow consumer.
> 
> Is there some documentation about this point ?

The documentation is lacking in this area :(

However, the principle is pretty simple: You can think of the network as 
series of tubes. Each tube has some max capacity. For example, when 
communicating using 2 0MQ sockets there's senders 0MQ queue, kernel 
SNDBUF, queue in the NIC, queue in the switch, queue in the receiver's 
NIC, kernel RCVBUF, receiver's 0MQ queue...

When consumer is slow these queues start growing. The process starts at 
the place of the bottleneck then propagates up back to the sender. Once 
all the queues are full, meaning that sender cannot send another message 
without exceeding the limits, some measure must be taken.

The exact measure depends on the messaging pattern. For example for 
REQ/REP send function blocks in such a situation. For PUB/SUB the 
message is accepted and dropped.

While the latter looks scary, it makes perfect sense because blocking 
would cause whole system to stop. Even innocent subscribers would not 
get messages.

If you want to reasonably guarantee the delivery, the only way is to 
make the queues very large. We've seen queues set to max. 200GB to be 
able to hold whole trading day of market data.

Martin