[zeromq-dev] Problem with zeromq and large messages

[Brant Merryman]

Sorry, I did miss that part of the documentation where it says that the return value is the number of bytes in the message which can be greater than the length parameter.

So, if I call it and the return code is greater than my buffer size – then I need to call it again – and then does it give me the next length bytes? Or have I already just lost those bytes?


From: "Bruno D. Rodrigues" <bruno.rodrigues at litux.org<mailto:bruno.rodrigues at litux.org>>
Reply-To: ZeroMQ development list <zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>>
Date: Friday, January 3, 2014 at 12:14 PM
To: "zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>" <zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>>
Subject: Re: [zeromq-dev] Problem with zeromq and large messages

read the man page ;)

recv returns the size of the message, but if the buffer you give is smaller than the message, zmq won’t write more bytes than the ones available.

the more flag is unrelated to the buffer size. It just means that multiple messages (parts) can be gathered together as a whole message by setting the more flag to on for all parts except the last one.


ALL: what are your thoughts about some api that could return the size of the message before calling the recv? I have the same problem with jeromq where I can’t pass a buffer into recv because I don’t know the message size, so I need to rely on byte[] recv() which will create a byte array for each message. Some kind of rc = zmq_recv_size(socket, flags) which would have no side effects, in the sense it could be called multiple times without taking that message out of the queue. Then we could adjust the buffer size and call the current recv(socket, buffer, flags)



On Jan 3, 2014, at 17:48, Brant Merryman <brantm at phmgmt.com<mailto:brantm at phmgmt.com>> wrote:

I have been using zeromq in my application and I’ve been having some problems with it.

Sometimes I need to send large messages. I have code that works kind of like this:

char eventBuffer[BUFFER_SIZE];

for (int more = 1; more; ) {
rc = zmq_recv(socket, eventBuffer, BUFFER_SIZE, 0);
if (rc > 0) {
<accumulate the data in the event buffer>
size_t moreSz = sizeof(int);
zmq_getsockopt(socket, ZMQ_RCVMORE, &more, &moreSz);
} else {
printf(“zmq_recv failed: %d\n”, rc);
more = 0;
}
}

<handle the accumulated data>


The process that is sending the data uses code like this:

rc = zmq_send(socket, data, data_length, 0);
if (rc <= 0) {
printf(“zmq_send failed: %d\n”, rc);
}

The problem is that when the data being sent is less than BUFFER_SIZE – no problem. But when the data sent is larger than BUFFER_SIZE, the zmq_recv call gets the same number of bytes that was sent. What I was expecting was that the number of bytes returned would be BUFFERSIZE and that zmq_getsockopt would result in more being 1 and we would repeat that data_length/BUFFER_SIZE times and then once more for the remaining data and then more would go to 0 and we would exit the loop.

How the heck is this supposed to work? What am I doing wrong?

Thanks.

Brant
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140103/f100b2ff/attachment.htm>