[zeromq-dev] Vector transmission via REQ-REP. Am I missing something obvious?

[Thomas Rodgers]

I did a talk on last year's CppCon which addresses in some detail the
general problem you are encountering -

https://www.youtube.com/watch?v=wbZdZKpUVeg

In general, you can't simply write the bytes of a struct (or class)
containing non-trivial (variable length or not) types as members into a zmq
buffer and hope for success. Even inproc, C++'s lifetime management rules
will eventually cause you to see unexpected results, when, for instance,
the rehydrated struct on the receiving end has a vector which points into
memory on the sending end which has already been free'd because the
containing struct went out of scope, and the vector member's destructor is
run. So, the advice to use a serialization library (or the Boost Fusion
approach from my talk if that makes sense) is sound.

As for the question about using vectors as buffers. It is fine, you can
reserve to the expected case size and resize if you get a larger message.
Whether or not a fixed buffer of the maximum expected size is more
wasteful, be aware there are pathological cases where your vector backed
buffer can end up being almost twice the size you intended. I frequently
use std::array<> as my buffer type and size it to the largest expected
message size at compile time. I frequently use stack-locals for synchronous
receive buffers.

Slightly OT - In C++, instances of structs *are* objects. There is little
practical difference between struct and class other than default member
visibility; the class will have private members by default, the struct will
have public members by default.


On Thursday, April 9, 2015, Dorvin <dorvin at tlen.pl> wrote:

> sizeof() for containers doesn't work the way you'd like to. You should
> use size() function of std::vector.
>
> As you're using C++ i would recommend to change structs to objects (even
> without methods if you don't need them), overload operators << and >>
> for this class and do some simple serialization/deserialization or use
> serialization library like Google Protobuf.
>
>
> Cheers,
> Jarek
>
>
>
> W dniu 2015-04-09 o 12:53, James Chapman pisze:
> > -- In case this appears twice, sent again this time from the list
> > subscribed account --
> >
> > Hello list
> >
> > I could use some help... Using the REQ-REP model in C++ I'm trying to
> > send a vector, but failing miserably and I can't work out why. Maybe
> > this will never work? Maybe the idea of sending a vector is a bad and
> > broken idea to start with and I should instead be trying to send a fixed
> > size? But then if sending a vector is impossible, how do I send a
> > variable amount of data? Should I just be using large enough buffers to
> > allow for my largest expected data transfer? -This seems wasteful.
> >
> > Guidance on this matter would be greatly appreciated.
> >
> > I should add that failure is happening on the receiving end when trying
> > to read the vector field.
> >
> > example (In this case TestMessage is a struct of what I want to send. If
> > I include a vector<int> field in TestMessage it fails.)
> >
> > zmq::message_t  * pZmqMsgOut =new  zmq::message_t(sizeof(TestMessage));
> > memcpy((void  *)pZmqMsgOut->data(), pTestMsg,sizeof(*pTestMsg));
> >
> >
> > Thanks in advance.
> > James
> >
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150409/1f279d6c/attachment.htm>