[zeromq-dev] Need advice on ZMQ_STREAM socket, zmq_recv truncating data

[Joe Lin]

I’m using ZMQ_STREAM socket talking to a TCP server. A simplified workflow is that the server will stream large amount of data upon client connection. The stream is composed of long sequence of various sized chunks.

The client code that reads the stream (omit the error checking):

char buf[BUF_SIZE];
int64_t more;
size_t more_size = sizeof(more);

while (true) {
  // first read the socket id
  int nbytes = zmq_recv(client_socket, buf, BUF_SIZE, 0);
  more = 1;

  // read the data
  while ( more ) {
    // read the data chunk, but data get truncated from time to time ??
    nbytes = zmq_recv(client_socket, buf, BUF_SIZE, 0); 

    //… do something with the received data in the buffer

    // more always return 0 ?? The second while loop is of no use
    zmq_getsockopt(client_socket, ZMQ_RCVMORE, &more, &more_size)
  }
}

I have two questions. First is that I find the data that I received from zmq_recv can get truncated. I know this is the intend behavior of zmq_recv. But I cannot think of a good way to avoid this unless I allocate a very large buffer to begin with. What is a good practice for this problem? 

Second is that I found that zmq_getsockop of ZMQ_RCVMORE always return more=0.  Why is that? I wonder how zmq_recv buffers data internally for ZMQ_STREAM socket. I would expect when dealing with a TCP stream, zmq_recv should fill the supplied buffer as much as possible (as opposed to truncating) and if there are still some more data to be read then zmq_getsockop of ZMQ_RCVMORE would return more=1.

Thanks for any help.