[zeromq-dev] Confirmation of ZMQ_PUB + ZMQ_SWAP intended behavior

Christian Martinez chmar at microsoft.com
Fri Aug 19 21:19:47 CEST 2011

Maybe I should ask this in a different way....Can you use swap to ensure that a subscriber who connects after a publisher has started will not miss messages.... If that's not the intended use what use is SWAP?


-----Original Message-----
From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Martin Sustrik
Sent: Wednesday, August 17, 2011 11:11 PM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Confirmation of ZMQ_PUB + ZMQ_SWAP intended behavior

On 08/17/2011 11:50 PM, Pieter Hintjens wrote:
> On Wed, Aug 17, 2011 at 11:38 PM, Ian Barber<ian.barber at gmail.com>  wrote:
>> If there is an identity on ZMQ (<  4) SUB, then the publisher will 
>> buffer up to the HWM. If the SWAP is set it will effectively extend 
>> that HWM up to it's size. If there's no identity, the HWM wont be 
>> hit, and nothing will be written to swap.
> You sure about this? I didn't realize the HWM was dependent on the SUB 
> setting an identity...

There are multiple issues mixed here. Let's deconstruct the whole thing:

1. There's a buffer on SUB side as well as on the PUB side. The pushback works as follows:
    a. If sender application sends faster than the network is able to transfer, the buffer on the send side starts filling. When it reaches HWM it enters the exceptional state and starts dropping messages.
    b. If messages are arriving at receiving application faster than the application is reading them, the buffer on the receive side starts filling. When it reaches HWM it enters the exceptional state and stops reading from the underlying TCP connection.
    c. If b. happens, TCP buffers on both receive and send side start filling up and ultimately the sender application won't be able to send more data to the network (TCP pushback). At that point a. kicks in and ultimately results in dropping messages.

    You can thing of the algorithm as a series of tubes of various diameters. If there's too much liquid poured into the tubes to smoothly flow through a particular portion of the tube, the liquid starts to accumulate upstream from the critical portion of the tube and eventually all the upstream tubes become full of water and you won't be able to pour any additional liquid in.

2. SWAP semantics are opaque. The limit of the buffer is expanded by the size of the swap. In other words, backpressure kicks in only after HWM + SWAP is reached.

3. IDENTITY has no effect on the backpressure mechanism. The only thing it does is that the "connection" is not cancelled when the peer dies. 
I.e. at the moment the receive peer dies the messages start accumulating in the sending peer (ultimately reaching HWM+SWAP and applying
backpressure) and start flowing only after the receiving peer is restarted.

As a side note, I would strongly suggest to set HWM to some sane finite value by default. Default infinite HWM is just confusing people. Compare TCP sockets where there's fixed default buffer size and nobody seems to have a problem with understanding the backpressure mechanism and 0mq with infinite default buffer size where the backpressure is constant source of confusion.

zeromq-dev mailing list
zeromq-dev at lists.zeromq.org

More information about the zeromq-dev mailing list