[zeromq-dev] Need advice on ZMQ_STREAM socket, zmq_recv truncating data

[Luca Boccassi]

On Wed, 2018-08-01 at 23:45 -0700, Joe Lin via zeromq-dev wrote:
> I’m using ZMQ_STREAM socket talking to a TCP server. A simplified
> workflow is that the server will stream large amount of data upon
> client connection. The stream is composed of long sequence of various
> sized chunks.
> 
> The client code that reads the stream (omit the error checking):
> 
> char buf[BUF_SIZE];
> int64_t more;
> size_t more_size = sizeof(more);
> 
> while (true) {
>   // first read the socket id
>   int nbytes = zmq_recv(client_socket, buf, BUF_SIZE, 0);
>   more = 1;
> 
>   // read the data
>   while ( more ) {
>     // read the data chunk, but data get truncated from time to time
> ??
>     nbytes = zmq_recv(client_socket, buf, BUF_SIZE, 0); 
> 
>     //… do something with the received data in the buffer
> 
>     // more always return 0 ?? The second while loop is of no use
>     zmq_getsockopt(client_socket, ZMQ_RCVMORE, &more, &more_size)
>   }
> }
> 
> I have two questions. First is that I find the data that I received
> from zmq_recv can get truncated. I know this is the intend behavior
> of zmq_recv. But I cannot think of a good way to avoid this unless I
> allocate a very large buffer to begin with. What is a good practice
> for this problem? 

zmq_recv, given it takes a limited size buffer from the application,
can't do much but truncate.
Use zmq_recv_msg which will fill the message with everything that is
received.

> Second is that I found that zmq_getsockop of ZMQ_RCVMORE always
> return more=0.  Why is that? I wonder how zmq_recv buffers data
> internally for ZMQ_STREAM socket. I would expect when dealing with a
> TCP stream, zmq_recv should fill the supplied buffer as much as
> possible (as opposed to truncating) and if there are still some more
> data to be read then zmq_getsockop of ZMQ_RCVMORE would return
> more=1.
> 
> Thanks for any help.

Read the zmq_socket manpage for details on what the multipart is used
for with the stream socket.

-- 
Kind regards,
Luca Boccassi