[zeromq-dev] Slow subscriber runs out of memory

[Martin Sustrik]

On 05/23/2011 05:17 PM, a frost wrote:

> I am evaluating ZMQ, doing some initial testing with a PUB-SUB
> configuration. The publisher and subscriber are in separate processes on
> the same machine. I am using TCP transport and have set HWM=10 on both
> sides.
>
> The subscriber is slow compared to the publisher and its memory usage
> increases rapidly as messages are received from the publisher.
>   Eventually, the subscriber process runs out of memory.
>
> I want messages to be discarded at the subscriber end if it cant process
> them fast enough, rather than memory usage increasing without bound as
> received messages are buffered. I thought setting HWM on the SUB side
> would give me this behaviour, but setting this has no effect on memory
> usage.

Yes. The messages should be discarded.

>
> Any help would be appreciated. Source code for my test programs is at:
>
> Publisher: http://pastebin.com/2YtRAi8C
> Subscriber: http://pastebin.com/Sr16RP75

The programs look OK to me. I guess there is a bug either in 0MQ or 
jzmq. I wonder if the same thing happens in C (ie. when jzmq is not used).

Martin