[zeromq-dev] memory leak in PUB/SUB sockets

[Daniel Sentenac]

Hello,

I am currently evaluating zeromq in PUB/SUB configuration for a video 
distribution system.
The server runs on Linux (Scientific Linux 2.6.23.9-i386-1.12-00 #3 SMP 
Wed Jun 24 10:34:37 CEST 2009 i686 athlon i386 GNU/Linux).
The server is written in C and basically does :
  /* Initialise 0MQ infrastructure with a single I/O thread */
  camroot->ctx = zmq_init (1);
  assert (camroot->ctx);
  /* Create a ZMQ_PUB socket to send video data. */
  camroot->s_video = zmq_socket (camroot->ctx, ZMQ_PUB);
  assert (camroot->s_video);
 
 /*Set high_water_mark = 1 to limit the queue size */
  rc = zmq_setsockopt (camroot->s_video, ZMQ_HWM, &high_water_mark,
               sizeof high_water_mark);
  assert (rc == 0);
 
while (1) {
    
/********************************************************************************/
    /* Get a new image frame and compress it in jpeg to produce 
compressed_size,compress_image data*/
    
/********************************************************************************/
  /*Make frame available in ZMQ queue*/
    rc = zmq_msg_init_size (&msg, compressed_size);
    if (rc != 0) {
      CfgMsgAddError ("aquire_and_publish_thread> error in 
zmq_msg_init_size: %s",
              zmq_strerror (errno));
      rtn = CFG_FAIL;
    }
   
    memcpy (zmq_msg_data (&msg), compressed_image, compressed_size);
    /* release compressed_image memory in this case*/
    if (compressed_image) {free(compressed_image);compressed_image=NULL;}

    //rc = zmq_msg_init_data (&msg, compressed_image, compressed_size,
    //            free_data, NULL); This is an alternative, works the same
    if (rc != 0) {
      CfgMsgAddError ("aquire_and_publish_thread> error in 
zmq_msg_init_data: %s",
              zmq_strerror (errno));
     rtn = CFG_FAIL;
    }
    rc = zmq_send (cam->s_video, &msg, 0);
    if (rc != 0) {
      CfgMsgAddError ("aquire_and_publish_thread> error in zmq_send: %s",
              zmq_strerror (errno));
     rtn = CFG_FAIL;
    }
    rc = zmq_msg_close (&msg);

}
The receiver clients run on a Linux similar platform. The client is 
written in Java with my own bindings since I couldn't make work the ones 
provided with in jzmq release (problem at run time to locate the 
function libraries, already reported in other mails, and according to me 
is not solved). Anyway, this is not the point here.
I run the server/client for long periods and appreciate the memory 
consumption with "top" tool, on both server and client sides.
1) The client uses a basic tcp zmq_connect to retreive the compressed 
images and works without any problem of memory leak.
2) If the server runs without any client connected, it works without 
memory leaks
3) Once a client or several are connected, memory leaks at server side 
occur. The behavior is the following:
     - The more clients are connected, the quicker the memory leak trend is
    - The leak depends on the client reactivity to the incoming 
messages. The slower is the client to treat messages, the quicker is the 
memory leak growing on the server side.

It really looks like the server is not able to control correctly the 
message queuing, even when using ZMQ_HWM set to 1.
More precisely, at the level of the zmq_msg_close function it is told that :
/Actual release of resources associated with the message object shall be
       postponed by 0MQ until all users of the message or underlying 
data buffer have
       indicated it is no longer required
/It seems that the problem is situated exactly at this level, and 
posponed message release are forgotten in some way. Any idea how to 
correct this ?

Thanks,

Daniel Sentenac