[zeromq-dev] idea for perf enhancement in user code

[Chuck Remes]

On Dec 14, 2011, at 6:20 AM, Mikko Koppanen wrote:

> On Wed, Dec 14, 2011 at 8:32 AM, Martin Sustrik <sustrik at 250bpm.com> wrote:
>> In theory we can add something like zmq_msg_add_ref() to get that
>> functionality.
>> 
>> The question is whether more explicit reference management using
>> separate zmq_msg_t per reference doesn't lead to better coding style.
> 
> Hi,
> 
> I would be hesitant to expose reference counting via the API as this
> is a very special case. Maybe we could rather have a send flag called
> ZMQ_DONTCLOSE or something similar so that message can be reused if
> necessary?

I like this idea better but with a slight change. For encapsulation, this property of "reusability" should be stored within the message and not be part of the socket API.

There has already been an earlier discussion to add a call to the message API called zmq_msg_flags(). This was intended to retrieve the RCVMORE and SNDLABEL (now reverted) flags from the message. I wrote the patch for it.

Why not add zmq_set_msg_flag() as well?

e.g.
  rc = zmq_set_msg_flag(msg, ZMQ_MSG_DONTCLOSE, 1); // set it

  rc = zmq_set_msg_flag(msg, ZMQ_MSG_DONTCLOSE, 0); // remove it

There might be a more elegant way of designing that particular API, but I don't remember C well enough to suggest it. I wanted to do "~ZMQ_MSG_DONTCLOSE" but that won't work. :\

It does *not* break backward compatibility as far as I can see. Internally, the socket calls a close() method on the message object, but the object can interrogate its own flags and know to skip the close logic when this flag is set. So, this would require *no change* in the socket. Isn't this what OO and encapsulation are all about? :)

So, hopefully I have addressed the issue of exposing the refcounting directly. This API change would allow for future extensions when we may add other message flags.

Lastly, allowing for a zmq_msg_t to stick around forever helps certain applications achieve zero-copy semantics when otherwise they could not. Some managed languages (like Ruby in my specific case) doesn't allow the programmer to pin memory in place for zero-copy. You have to allocate native memory in the heap and then dealloc it when you are done. By allowing a message to stick around forever, I can do this allocation a single time and let 0mq do the scatter-gather send for me.

I'll wait a few more days for comments and then write a patch and submit it.

cr