[zeromq-dev] ZMQ - Send std::shared_ptr via inproc pub-sub?

BJovke bjovan at gmail.com
Wed Jun 24 22:51:40 CEST 2020


That's a good solution.

But what happens if receiver never receives the message or doesn't take the
shared_ptr out of the table for some other reason?

It will be there forever.

Yes, you're right, sending a pointer to shared_ptr is not good because
message is sent asynchronously. It could be done by finding a way to know
that the message has been received.

On Wed, Jun 24, 2020, 22:31 Thomas Rodgers <rodgert at twrodgers.com> wrote:

>
>
> On Jun 24, 2020, at 12:16 PM, Serguei Khasimkhanov <SerKhas at hotmail.com>
> wrote:
>
> 
> Thank you for the answers. I will reply to everyone in this email:
>
>
> > BJovke: You can send a pointer to your shared_ptr and then the receiver
> should copy construct (it's not an actual copy but a reference) it's own
> shared_ptr from it and forget about this pointer.
>
> But the problem is that the original shared pointer may get destroyed or
> replaced before the receiver can make a copy.
>
>
> > Rodrigo: If you can change the class definition, you could use the
> seldom seen std::enable_shared_from_this.
>
> If I send a pointer to the object, and the original shared_ptr to the
> object goes out of scope, the object will be destroyed, and I won't be able
> to `std::enable_shared_from_this`.
>
>
> > Sergei: You should serialize your object on publisher side -> send ->
> receive -> deserialize on subscriber. By object I mean target object on
> which shared_ptr points.
>
> If I serialize a shared_ptr, it won't get copy-constructed, so the
> reference count won't increase, and the owned pointer could get freed
> before any receivers receive the message.
>
>
> > Rafal: Don't you think that sending a pointer to a memory is very error
> prone? e.g.: object may be destroyed before the receiver gets a serialized
> copy of shared_ptr as ref_count will not consider the fact that there is an
> instance of pointer in the ZMQ inproc buffer. IMO it would be better to
> send an object identifier which could be used by a receiver to retrieve
> data from some data provider.
>
> Yes that's what I'm worried about. I don't know when to delete the pointer
> because I don't know when everyone is done using it. That data provider
> idea is my backup plan - if a receiver wants a copy of the data, they send
> a separate data request to the publisher (through a separate socket), and
> the publisher sends each one a properly-constructed copy of the
> std::shared_ptr.
>
>
> I have code which has done this. As has been pointed out, this is tricky
> as you need to ensure the lifetime of the thing being pointed to until the
> receiver has received the message and taken ownership of the shared
> reference.
>
> The way I ended up managing it was storing the shared_ptr in a table
> hashed by the pointer value (each hash bucket having it’s own mutex to
> reduce contention) and then sending the pointer value over the inproc
> socket as a key into the table. Retrieving the key then moves the
> shared_ptr out of the table so that the receiver has ownership of the
> shared_ptr ref.
>
>
> > Bill: Your original question implies (but does not state explicitly)
> that you want to use 0mq to send shared_ptr’s to other threads in the same
> process.    If that is the case you need to be cognizant of (
> https://en.cppreference.com/w/cpp/memory/shared_ptr):
>
> Sorry for the confusion, yes it is all the same process. My goal is to
> have each subscriber to copy-construct their own copy of the shared_ptr.
>
>
> > Harald: shared_pointers are thread safe  (not the underlying data). so
> you can copy the shared_ptr to the thread, read the date, and forget about
> it. no need to try some tricks like serialize pointer addresses, you will
> not succeed that way anyway.
>
> The problem is, I cannot just "copy the shared_ptr and forget about it".
> If it goes out of scope before any subscribers make a copy, its data will
> be deallocated.
>
>
> ------------------------------
> *From:* zeromq-dev <zeromq-dev-bounces at lists.zeromq.org> on behalf of
> Harald Achitz <harald.achitz at gmail.com>
> *Sent:* June 24, 2020 2:36 PM
> *To:* ZeroMQ development list <zeromq-dev at lists.zeromq.org>
> *Subject:* Re: [zeromq-dev] ZMQ - Send std::shared_ptr via inproc pub-sub?
>
> shared_pointers are thread safe  (not the underlying data)
> so you can copy the shared_ptr to the thread, read the date, and forget
> about it
> no need to try some tricks like serialize pointer addresses, you will not
> succeed that way anyway
>
> On Wed, 24 Jun 2020 at 20:25, Bill Torpey <wallstprog at gmail.com> wrote:
>
> Serguei:
>
> Your original question implies (but does not state explicitly) that you
> want to use 0mq to send shared_ptr’s to other threads in the same process.
>    If that is the case you need to be cognizant of (
> https://en.cppreference.com/w/cpp/memory/shared_ptr):
>
> The ownership of an object can only be shared with another shared_ptr by
> copy constructing or copy assigning its value to another shared_ptr.
> Constructing a new shared_ptr using the raw underlying pointer owned by
> another shared_ptr leads to undefined behavior.
>
>
>
> If that is not the case (i.e., if the destination of the send is not
> another thread in the same process), you need to actually serialize the
> data, as suggested by *Sergei Nikulov <sergey.nikulov at gmail.com
> <sergey.nikulov at gmail.com>>*
>
> Which is it?
>
> B
>
> On Jun 24, 2020, at 2:06 PM, Rafał Dudycz <rafal.dudycz at gmail.com> wrote:
>
> Hi,
>
> Don't you think that sending a pointer to a memory is very error prone?
> e.g.: object may be destroyed before the receiver gets a serialized copy of
> shared_ptr as ref_count will not consider the fact that there is an
> instance of pointer in the ZMQ inproc buffer.
> IMO it would be better to send an object identifier which could be used by
> a receiver to retrieve data from some data provider.
>
> --
> Rafal
>
>
> śr., 24 cze 2020 o 19:56 Sergei Nikulov <sergey.nikulov at gmail.com>
> napisał(a):
>
> On Wed, Jun 24, 2020 at 8:11 PM Serguei Khasimkhanov
> <SerKhas at hotmail.com> wrote:
> >
> > Hello all,
> >
> > I have a publisher thread that has a bunch of shared_ptrs to large
> chunks of data. Occasionally I want to share some of those shared_ptrs with
> subscribers (for read-only.) The problem is that ZeroMQ only allows sending
> binary data, not C++ objects. How can I accomplish this?
>
> You should serialize your object on publisher side -> send -> receive
> -> deserialize on subscriber.
> There is no other way except you doing some interprocess communication.
> Ref. https://www.google.com/search?q=c%2B%2B+serialization
>
> >
> > I was thinking I could allocate a copy of the shared_ptr on the heap,
> then publish a pointer to it, but since I don't know when the subscribers
> are done using that pointer, I don't know when to deallocate it.
> >
> > - Ser
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> --
> Best Regards,
> Sergei Nikulov
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20200624/e50c22f3/attachment.htm>


More information about the zeromq-dev mailing list