[zeromq-dev] How to get buffer out of zmq_msg

[john skaller]

On 08/02/2012, at 11:09 AM, Martin Sustrik wrote:

> Hi John,
> 
>> If you initialise the message empty, zmq_recvmsg will allocate
>> exactly the right amount of space for you. This is ideal.
>> But there's no way to get the buffer out of the msg, you have
>> to copy it. This is wastes a memcpy, and wastes storage temporarily.
>> You have to copy it because zmq_close will deallocate the buffer.
> 
> The problem is that 0mq messages are reference counted, so you may be 
> sharing the message buffer with others.

A solution for zmq_release function would be:

(a) if the buffer isn't shared, return it, and clear the msg_t object
  (i.e. set pointer to NULL)

(b) if the reference count is 2 or more:
   copy the buffer, decrement the count,
   clear the msg_t object


> 
>> Of course .. you can *cheat* and simply not close the zmq_msg_t,
>> and in the current implementation I am guessing this would work.
> 
> Yes. That should work.

OK, I have committed zmq_recvmmsg to my clone now. 
Not sure about the name!

This is as close to an XSI compliant
interface as I can think of. This function is for receiving 0MQ multi-part
messages only. If used on a non-multi-part message it just reads a single
message. 

The semantics differ from XSI readv() function in that the 0MQ variant
allocates and sets the length of the buffers according to the 
0MQ multi-part message part length.

If the vector is too short to hold all the message parts, the function
simply returns with ZMQ_RCVMORE set on the socket,
therefore the behaviour is identical to zmq_recv if you 
set *count_=1. 

Note carefully the assumptions, please let me know if any of these
assumptions are wrong in the current implementation, if there's
a better way, or whatever. The fragility is unfortunate and can easily
be avoided by copying the buffers but this is inefficient.

(a) It is safe to simply forget a zmq_msg we allocated and initialised,
and filled in by zmq_recv. There cannot be any sharing.

(b) zmq_recv allocates a msg buffer with malloc family function so it 
is safe to allow the client to free these buffers with free()


// Receive a multi-part message
// 
// Receives up to *count_ parts of a multi-part message.
// Sets *count_ to the actual number of parts read.
// ZMQ_RCVMORE is set to indicate if a complete multi-part message was read.
// Returns number of message parts read, or -1 on error.
//
// Note: even if -1 is returned, some parts of the message
// may have been read. Therefore the client must consult
// *count_ to retrieve message parts successfully read,
// even if -1 is returned.
//
// The iov_base* buffers of each iovec *a_ filled in by this 
// function may be freed using free().
//
// Implementation note: We assume zmq::msg_t buffer allocated
// by zmq::recvmsg can be freed by free().
// We assume it is safe to steal these buffers by simply
// not closing the zmq::msg_t.
//
int zmq_recvmmsg (void *s_, iovec *a_, size_t *count_, int flags_)

--
john skaller
skaller at users.sourceforge.net