[zeromq-dev] Task queues using ZeroMQ and inproc

[Lindley French]

You can't avoid shared memory semantics *and* serialization. Serialization
is the only way to avoid some kind of shared memory.

That said, shared memory isn't a problem if all of your messages are
immutable after construction. I recommend using this approach whenever
possible, whether you use zmq for your message passing or not.

Unfortunately you can't safely pass C++ smart pointers through inproc
because they require object semantics to work, and inproc only provides
byte semantics. If there's a way around this I haven't found it yet.


On Wed, Mar 12, 2014 at 2:20 AM, Pieter Hintjens <ph at imatix.com> wrote:

> Passing pointers is fine if you are sure there won't be shared access.
> It's a decent pattern; to pass an object from one thread to another
> where it can be used and then destroyed.
>
> On Wed, Mar 12, 2014 at 9:35 AM, Cosmo Harrigan
> <cosmo.harrigan at singularityu.org> wrote:
> > Justin, thanks for the suggestion.
> >
> > I was sort of hoping to get some responses telling me why this isn't a
> good
> > approach, and why other approaches are preferable. If anyone has
> thoughts,
> > please let me know.
> >
> > Thanks,
> > Cosmo
> >
> >
> > On Tue, Mar 11, 2014 at 4:40 PM, Justin Karneges <justin at affinix.com>
> wrote:
> >>
> >> If you're using C++, you should be able to get away with using "smart
> >> pointer" approaches.
> >>
> >> I'm personally using Qt which has the best smart pointer approach I've
> >> ever seen (QSharedData). I use inproc sockets to send/recv pointers, but
> >> the application only has to deal with value objects. It's some wild C++
> >> voodoo.
> >>
> >> On 03/11/2014 03:25 PM, Cosmo Harrigan wrote:
> >> > Hi,
> >> >
> >> > If an application is using ZeroMQ exclusively as a concurrency
> >> > framework, and only using inproc sockets, and the goal is to
> implement a
> >> > task parallelism abstraction like the ventilator/sink pattern, is
> there
> >> > any way to avoid the overhead involved in object serialization and
> >> > deserialization that is involved since the approach is no longer
> using a
> >> > shared memory model with pointers to the objects?
> >> >
> >> > I see there was a similar discussion previously:
> >> > http://comments.gmane.org/gmane.network.zeromq.devel/7300
> >> >
> >> > which suggested passing object pointers as messages over inproc.
> >> > However, I am also concerned that if you do that, you're giving up one
> >> > of the main benefits of using message passing, since once again you
> have
> >> > the potential for multiple threads to share state. I also understand
> >> > that you're restricting yourself from scaling to additional machines.
> >> >
> >> > Also, Pieter had commented "don't pass pointers around" here:
> >> > http://osdir.com/ml/zermq-development/2013-05/msg00104.html
> >> >
> >> > What are common approaches to address this scenario?
> >> >
> >> > Thanks,
> >> > Cosmo
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > zeromq-dev mailing list
> >> > zeromq-dev at lists.zeromq.org
> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >
> >>
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140312/9b0b71ef/attachment.htm>