[zeromq-dev] Extending zmq_msg_t API
Thomas Rodgers
rodgert at twrodgers.com
Sat Jul 12 16:04:03 CEST 2014
Pull request submitted.
On Fri, Jul 11, 2014 at 7:43 AM, Thomas Rodgers <rodgert at twrodgers.com>
wrote:
> I'm on my annual road trip, so it will be few days still before I have
> time to put the patch together.
>
>
> On Friday, July 11, 2014, Goswin von Brederlow <goswin-v-b at web.de> wrote:
>
>> On Wed, Jul 09, 2014 at 04:53:16PM -0500, Thomas Rodgers wrote:
>> > Right. This is my trepidation about surfacing the refcount. The
>> > "sharedness" is indicated by the flags field on the msg type, and that,
>> I'm
>> > reasonably sure, is not altered once it is set.
>>
>> At least it can't only become unshared in the background, not suddenly
>> start being shared. With 2 copies of a message floating around one can
>> be closed or shared again inbetween the check and the copy. But with
>> only one copy (the one YOU hold) nobody else can share the message in
>> the background. Assuming you don't share message pointer between
>> threads.
>>
>> A zmq_msg_get(&msg, ZMQ_SHARED) is easy to add and should be thread
>> save, erring on sometimes returning true when a message is later not
>> shared anymore.
>>
>> Looking forward to a PULL request for that.
>>
>> MfG
>> Goswin
>>
>> > On Wed, Jul 9, 2014 at 4:05 PM, KIU Shueng Chuan <nixchuan at gmail.com>
>> wrote:
>> >
>> > > Couldn't the refcount change after you have obtained its value?
>> > > E.g.
>> > > Make a copy
>> > > Send the 1st
>> > > Read the refcount (2)
>> > > Background io releases 1st copy
>> > > On 9 Jul 2014 18:21, "Thomas Rodgers" <rodgert at twrodgers.com> wrote:
>> > >
>> > >> zmq_msg_get() could be extended to give either the refcount or an
>> > >> indicator on whether a message was share; based on other refcounted
>> designs
>> > >> I'm hesitant to promote surfacing the actual count. Similarly,
>> > >> zmq_msg_set() could allow 'unsharing' by adding a ZMQ_SHARED property
>> > >> #define and setting it's value to 0 (no effect on non-shared
>> messages).
>> > >>
>> > >> So the only API surface area change is an additional message
>> property.
>> > >> This seems the cleanest to me.
>> > >>
>> > >> On Wednesday, July 9, 2014, Goswin von Brederlow <goswin-v-b at web.de>
>> > >> wrote:
>> > >>
>> > >>> On Tue, Jul 08, 2014 at 10:42:41AM -0500, Thomas Rodgers wrote:
>> > >>> > tl;dr; Is there any objection to adding some sort of accessor to
>> the
>> > >>> API to
>> > >>> > determine if a given zmq_msg_t is_shared()?
>> > >>> >
>> > >>> > Background/Rationale:
>> > >>> >
>> > >>> > Something I encountered while writing a "high level" C++ wrapper
>> for
>> > >>> > zmq_msg_t and it's API is the following set of behaviors -
>> > >>> >
>> > >>> > zmq_msg_init(&msg_vsm, 20);
>> > >>> >
>> > >>> > Results in a type_vsm message, the body of which is held entirely
>> > >>> within
>> > >>> > the space allocated to zmq_msg_t
>> > >>> >
>> > >>> > zmq_msg_init(&msg_lmsg, 1024);
>> > >>> >
>> > >>> > Results in a type_lmsg message, the body is held as a reference
>> to a
>> > >>> block
>> > >>> > of size bytes.
>> > >>> >
>> > >>> > memcpy(zmq_msg_data(&msg_vsm), "VSM", 3);
>> > >>> > memcpy(zmq_msg_data(&msg_lmsg), "LMSG", 4);
>> > >>> >
>> > >>> > So far so good. Now copy -
>> > >>> >
>> > >>> > zmq_msg_copy(&msg_vsm2, &msg_vsm);
>> > >>> > zmq_msg_copy(&msg_lmsg2, &msg_lmsg);
>> > >>> >
>> > >>> > Now change contents -
>> > >>> >
>> > >>> > memcpy(zmq_msg_data(&msg_vsm2), "vsm", 3);
>> > >>> > memcpy(zmq_msg_data(&msg_lmsg2), "lmsg", 4);
>> > >>> >
>> > >>> > assert(memcmp(&msg_vsm, &msg_vsm2, 3) != 0); // ok
>> > >>> > assert(memcmp(&msg_lmsg, &msg_lmsg2, 4) != 0); // fail
>> > >>> >
>> > >>> > This happens by design (lmsg's are refcounted on copy, not deep
>> > >>> copied).
>> > >>> > But it results in a situation where a zmq_msg_t is sometimes a
>> Value
>> > >>> and
>> > >>> > sometimes a Reference. This could lead to astonishment for the
>> unwary.
>> > >>> >
>> > >>> > >From the perspective of a wrapper (particularly one that takes a
>> > >>> strong
>> > >>> > stand on value semantics and local reasoning), this behavior is
>> > >>> ungood. So
>> > >>> > my options are deep copy always or implement copy-on-write.
>> > >>> >
>> > >>> > For efficiency I prefer the latter approach in the case of
>> type_lmsg
>> > >>> > messages. I have implemented the copy-on-write logic through a
>> > >>> horrible
>> > >>> > brittle hack that examines the last byte of zmq_msg_t. I would
>> prefer
>> > >>> a
>> > >>> > less brittle solution.
>> > >>>
>> > >>> "lmsg's are refcounted on copy" Can't you access the refcount?
>> > >>> Or is that the API call you want to add?
>> > >>>
>> > >>> Maybe instead of is_shared() an unshare() call would be more
>> usefull,
>> > >>> which would copy the message payload if it is shared. Or both?
>> > >>>
>> > >>> MfG
>> > >>> Goswin
>> > >>> _______________________________________________
>> > >>> zeromq-dev mailing list
>> > >>> zeromq-dev at lists.zeromq.org
>> > >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> > >>>
>> > >>
>> > >> _______________________________________________
>> > >> zeromq-dev mailing list
>> > >> zeromq-dev at lists.zeromq.org
>> > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> > >>
>> > >>
>> > > _______________________________________________
>> > > zeromq-dev mailing list
>> > > zeromq-dev at lists.zeromq.org
>> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> > >
>> > >
>>
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140712/40cfb190/attachment.htm>
More information about the zeromq-dev
mailing list