[zeromq-dev] Ever growing memory even with HWM

[Chuck Remes]

On Jun 24, 2011, at 10:38 AM, Nam-Luc Tran wrote:

> Hello everyone,
> 
> Following is a simple publisher-subscriber java code, which represents the behaviour of one of our applications. When we launch the publisher on one machine and five subscribers on an other machine, memory on the publisher side keep growing and growing until eventually crashing. 
> 
> I believe we are in the "slow subscriber case" described in the zguide.
> 
> My first question is: isn't the publisher by default (no semi transient socket) supposed to drop every message if the subscriber is not ready? So where could that memory growing come from?
> 
> I have also tried with setting an identity to the subscriber and a HWM on the publisher as described in the zguide but the same memory problem occurs.
> 
> We use zeromq 2.1.7 with the latest stable java bindings on Ubuntu 10.04 64bits.
> 
> Any help will be appreciated,
> 
> Nam-Luc Tran
> 
> /////Publisher code/////
> 
> package com.ena.zmq;
> 
> import org.zeromq.ZMQ;
> 
> public class PubClient {
> 	public static void main(String[] args) {
> 	
> 		ZMQ.Context context = ZMQ.context(1);
> 		ZMQ.Socket publisher = context.socket(ZMQ.PUB);
> 		publisher.bind("tcp://*:5800");
> 		publisher.setHWM(20);
> 			
> 		byte[] msg = new byte[5000];
> 		msg[msg.length-1]=0;
> 		
> 		System.out.println("Publisher online");
> 		while(true){
> 			publisher.send(msg,0);
> 			try {
> 				Thread.sleep(10);
> 			} catch (InterruptedException e) {
> 				// TODO Auto-generated catch block
> 				e.printStackTrace();
> 			}
> 		}
> 		
> 	}
> }

To answer your first question, no. Without an HWM set, the memory use by a PUB socket is unbounded. It will only drop messages if there are no SUB sockets connected at all. As soon as one SUB socket connects, even if it is slow, the memory usage will grow if the SUB socket can't keep up.

Secondly, there is an error in your publisher code when setting the HWM. Take a look at the man page for zmq_setsockopt().

http://api.zeromq.org/2-1:zmq-setsockopt

All settings (except for 3) *only* take effect before calling zmq_connect()/zmq_bind(). So, move the publisher.setHWM() call to before the bind and try again.

cr

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110624/6d043019/attachment.htm>