[zeromq-dev] Problem with zeromq and large messages

Lindley French lindleyf at gmail.com
Fri Jan 3 19:33:50 CET 2014


My first thought is to prefer zmq_msg_recv() if you want to allocate a
buffer large enough for each message.

Now, if you're trying to be clever and only reallocate when the next
message is bigger than any you've seen so far, maybe there would be some
benefit to a new API method.


On Fri, Jan 3, 2014 at 1:14 PM, Bruno D. Rodrigues <
bruno.rodrigues at litux.org> wrote:

> read the man page ;)
>
> recv returns the size of the message, but if the buffer you give is
> smaller than the message, zmq won’t write more bytes than the ones
> available.
>
> the more flag is unrelated to the buffer size. It just means that multiple
> messages (parts) can be gathered together as a whole message by setting the
> more flag to on for all parts except the last one.
>
>
> ALL: what are your thoughts about some api that could return the size of
> the message before calling the recv? I have the same problem with jeromq
> where I can’t pass a buffer into recv because I don’t know the message
> size, so I need to rely on byte[] recv() which will create a byte array for
> each message. Some kind of rc = zmq_recv_size(socket, flags) which would
> have no side effects, in the sense it could be called multiple times
> without taking that message out of the queue. Then we could adjust the
> buffer size and call the current recv(socket, buffer, flags)
>
>
>
> On Jan 3, 2014, at 17:48, Brant Merryman <brantm at phmgmt.com> wrote:
>
>  I have been using zeromq in my application and I’ve been having some
> problems with it.
>
>  Sometimes I need to send large messages. I have code that works kind of
> like this:
>
>  char eventBuffer[BUFFER_SIZE];
>
>  for (int more = 1; more; ) {
> rc = zmq_recv(socket, eventBuffer, BUFFER_SIZE, 0);
> if (rc > 0) {
> <accumulate the data in the event buffer>
> size_t moreSz = sizeof(int);
> zmq_getsockopt(socket, ZMQ_RCVMORE, &more, &moreSz);
> } else {
> printf(“zmq_recv failed: %d\n”, rc);
> more = 0;
> }
> }
>
>  <handle the accumulated data>
>
>
>
>  The process that is sending the data uses code like this:
>
>  rc = zmq_send(socket, data, data_length, 0);
> if (rc <= 0) {
> printf(“zmq_send failed: %d\n”, rc);
> }
>
>
>  The problem is that when the data being sent is less than BUFFER_SIZE –
> no problem. But when the data sent is larger than BUFFER_SIZE, the zmq_recv
> call gets the same number of bytes that was sent. What I was *expecting* was
> that the number of bytes returned would be BUFFERSIZE and that
> zmq_getsockopt would result in *more* being 1 and we would repeat that
> data_length/BUFFER_SIZE times and then once more for the remaining data and
> then more would go to 0 and we would exit the loop.
>
>  How the heck is this supposed to work? What am I doing wrong?
>
>  Thanks.
>
>  Brant
>  _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140103/6b6b7ff0/attachment.htm>


More information about the zeromq-dev mailing list