[zeromq-dev] Communication between C++ zmq client and Python zmq Server

Thomas Rodgers rodgert at twrodgers.com
Wed Jan 31 00:13:56 CET 2018


You can’t memcpy a std::string, and in fact, you should t be using memcpy
in general in C++. I won’t go into the why of all of that here, but start
by changing -

memcpy(request.data(), *&result*, 30);

To

memcpy(request.data(), *result.c_str()*, result.size() + 1);


On Tue, Jan 30, 2018 at 4:46 PM Bernardo Augusto García Loaiza <
botibagl at gmail.com> wrote:

> Hi, I again.
>
> I've performed some small test in relation to the way on how to I sending
> the data member structure.
>
> I am turning the int data member structure to string before to be send it
> ..
> In the following code section  I am turn btnState in string data and
> assign it to string s variable and I contatenated it some additional string
> content and
> I sending this string concatenated result:
>
> void ZMQComponent::instrumentDataSend(instrumentData a){
>     /*  Initialize the data members structure instrumentData */
>     a.pos = sofa::defaulttype::Vec3d(1.0f, 1.0f, 1.0f);
>     a.quat = defaulttype::Quat(1.0f, 1.0f, 4.0f, 1.0f);
>     a.btnState = 5671;
>     a.openInst = 1.0f;
>     a.blnDataReady = false;
>
>
>
>    * string s, test, result, d;
>     s = to_string(a.btnState);
>     test = " is a number";
>     result = s + test;*
>
>
>
>     /*  We send  the btnState data  */
>
>
>     zmq::message_t request(30);
>
>
>
>
> /*  We ask for the memory address to ge the btnState content and send it. */
>
>
>     memcpy(request.data(), *&result*, 30);
>     socket.send(request);}
>
>
> The output or the message which arrive to python zmq server is the content
> of
> result variable (btnState turn to string in s content variable + string
> test concatenated)
> and some symbols characters of the :
>
> (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python server.py
> ZMQ Server listening ...
>
> *Received message from Sofa: b'\xb0\x1d\x19\xf4\xfd\x7f\x00\x00\x0e\x00\x00\x00\x00\x00\x00\x0045 is a number'*
>
> If I assign a value less than 30 in the request, by example zmq::message_t
> request(10) the output in my server is:
>
> Received message from Sofa: b'\x90\x94\xa1\x00\xfc\x7f\x00\x00\x0e\x00'
>
> If I assign a value greater than 10 in the request, by example zmq::message_t
> request(20) the output in my server is:
>
> Received message from Sofa: b'\x80$(\xc7\xfc\x7f\x00\x00\x0e\x00\x00\x00\x00\x00\x00\x00*45 i*
>
> Then, the string or object which I receive in the server side, it has as
> long as the length or size assigned to zmq::message_t request variable
>
>
> Based in the above mentioned, is ZMQ whom add this strings in my message
> received?
>
> According to the previous process, my message is arrived to my server,
>
> then ....
>
>
>  is correct attempt what the serialization process with some entity like
> protocol buffer is necessary?
>
>
>  I understand that use something like google protocol buffer allow have
> some correlation more controlled in relation to objects sent and the
> objects received in relation of their real content ...
>
> In any case, how to can I remove the strings or characters symbols that
> are added in the message arrived to the server?
>
>
>
>
>
>
>
>
> Bernardo Augusto García Loaiza
> Ingeniero de Sistemas
> Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT
> http://about.me/bgarcial
>
>
> On Sat, Jan 27, 2018 at 11:24 AM, Thomas Rodgers <rodgert at twrodgers.com>
> wrote:
>
>> “strat team” was supposed to be std::stringstream (Thanks for nothing
>> autocorrect!)
>>
>> http://en.cppreference.com/w/cpp/io/basic_stringstream
>>
>> On Sat, Jan 27, 2018 at 10:22 AM Thomas Rodgers <rodgert at twrodgers.com>
>> wrote:
>>
>>> Strings are the simplest. If you don’t need to parse the data on the C++
>>> side, just format it as a JSON map on the C++ side, stream it into a strat
>>> team, send the resulting string on the socket and use Python’s support for
>>> parsing JSON on the received strong. For instance.
>>>
>>> You can also send structured binary to Python, as I mentioned; the
>>> Python CTypes module (included with Python) will let you define accessors
>>> for that data. As long as the sending and receiving side are the same CPU
>>> architecture you don’t have to worry about endianess conversions.
>>>
>>> On Sat, Jan 27, 2018 at 9:13 AM Bernardo Augusto García Loaiza <
>>> botibagl at gmail.com> wrote:
>>>
>>>> Hi Luca, this mean, then, with structured string or data is necessary
>>>> some middleware entity like GPB such as Thomas tell us ...
>>>>
>>>>
>>>> Bernardo Augusto García Loaiza
>>>> Ingeniero de Sistemas
>>>> Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT
>>>> http://about.me/bgarcial
>>>>
>>>>
>>>> On Sat, Jan 27, 2018 at 10:07 AM, Luca Boccassi <
>>>> luca.boccassi at gmail.com> wrote:
>>>>
>>>>> Yes you can just send unstructured binary data or strings - Python has
>>>>> native helpers for strings, and CZMQ as well
>>>>>
>>>>> On Fri, 2018-01-26 at 21:27 -0500, Bernardo Augusto García Loaiza
>>>>> wrote:
>>>>> > Thomas, I understand perfectly your explanation. But I want share
>>>>> > with you
>>>>> > another similar case, just for curiosity or to receive some
>>>>> > additional
>>>>> > orientation about it.
>>>>> >
>>>>> > I have a script client C++ ZEROMQ  as a documentation, and I have
>>>>> > script
>>>>> > server python, together based in the documentation zeromq website.
>>>>> > This mean, together are scripts basics
>>>>> >
>>>>> > From C++ client I am sending the string message
>>>>> > <
>>>>> https://github.com/bgarcial/zeroMQ_Client/blob/master/client.cpp#L27
>>>>> > >  and
>>>>> > I have a python server which receive this string message client
>>>>> > <https://github.com/bgarcial/zeroMQ_Client/blob/master/Python/server
>>>>> .
>>>>> > py#L20>
>>>>> > of  a direct way and show me their content
>>>>> >
>>>>> > In this basic sample (documentation zmq based) I am not
>>>>> > using  structures
>>>>> > data  members. Is for this reason that in this case I receive and I
>>>>> > can see
>>>>> > the content of the client message in the zmq server side without have
>>>>> > use
>>>>> > something like Google Protocol Buffer, This mean, is because in this
>>>>> > case I
>>>>> > am not sending any structures data  members?
>>>>> >
>>>>> > I appreciate one more your orientation
>>>>> >
>>>>> >
>>>>> > Bernardo Augusto García Loaiza
>>>>> > Ingeniero de Sistemas
>>>>> > Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT
>>>>> > http://about.me/bgarcial
>>>>> >
>>>>> >
>>>>> > On Fri, Jan 26, 2018 at 7:44 AM, Thomas Rodgers <
>>>>> rodgert at twrodgers.co
>>>>> > m>
>>>>> > wrote:
>>>>> >
>>>>> > > Short answer yes, it’s related to different languages. TCP and
>>>>> > > ZeroMQ only
>>>>> > > deal with transport and framing of messages (respectively) Python
>>>>> > > has no
>>>>> > > idea that that 10 bytes corresponds to a C/C++ struct of a given
>>>>> > > layout
>>>>> > > until you tell it. Different platforms (CPU architectures) have
>>>>> > > potentially
>>>>> > > different representations of fundamental types.
>>>>> > >
>>>>> > > On Thu, Jan 25, 2018 at 4:37 PM Bernardo Augusto García Loaiza <
>>>>> > > botibagl at gmail.com> wrote:
>>>>> > >
>>>>> > > > One more question. Why is necessary the serialization process? It
>>>>> > > > is
>>>>> > > > assumed that zeromq works via TCP and in the TCP protocol, the
>>>>> > > > data does
>>>>> > > > not undergo any transformation? This is related with the fact of
>>>>> > > > that I
>>>>> > > > have different platform/languages?
>>>>> > > >
>>>>> > > > On Thu, Jan 25, 2018 at 4:45 PM Bernardo Augusto García Loaiza <
>>>>> > > > botibagl at gmail.com> wrote:
>>>>> > > >
>>>>> > > > > Hi Thomas,
>>>>> > > > > Thanks for your illustrative response
>>>>> > > > >
>>>>> > > > > I'll look Google Protocol Buffers. My sender is from C++
>>>>> > > > > language and
>>>>> > > > > my reception is Python. What sort of installation recommend me
>>>>> > > > > you?
>>>>> > > > > Binaries or build protocol buffer along my C++ runtime? or o
>>>>> > > > > build protoc
>>>>> > > > > binary from source?
>>>>> > > > >
>>>>> > > > > On Wed, Jan 24, 2018 at 8:36 PM Thomas Rodgers <rodgert at twrodge
>>>>> > > > > rs.com>
>>>>> > > > > wrote:
>>>>> > > > >
>>>>> > > > > > You can have a look at Python’s ctypes module, which will let
>>>>> > > > > > you
>>>>> > > > > > define a ‘struct’ from Python with the same layout as your
>>>>> > > > > > C++ struct.
>>>>> > > > > >
>>>>> > > > > > You can also investigate any number of serialization
>>>>> > > > > > libraries that
>>>>> > > > > > have C++ and Python support, eg ProtoBufs or Thrift, or
>>>>> > > > > > MagPack or whatever.
>>>>> > > > > >
>>>>> > > > > > On Wed, Jan 24, 2018 at 5:26 PM Bernardo Augusto García
>>>>> > > > > > Loaiza <
>>>>> > > > > > botibagl at gmail.com> wrote:
>>>>> > > > > >
>>>>> > > > > > > Hi, ZMQ people.
>>>>> > > > > > > Greetings.
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > I have a  C++ zeromq client process in which I am sending
>>>>> > > > > > > some data
>>>>> > > > > > > members structures
>>>>> > > > > > >
>>>>> > > > > > > *ZMQComponent.h* file
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > #include <zmq.hpp>
>>>>> > > > > > > #include <sofa/defaulttype/VecTypes.h>
>>>>> > > > > > >
>>>>> > > > > > > // To Quat datatype
>>>>> > > > > > > #include <sofa/defaulttype/Quat.h>
>>>>> > > > > > > using sofa::defaulttype::Quat;
>>>>> > > > > > >
>>>>> > > > > > > using std::string;
>>>>> > > > > > >
>>>>> > > > > > > namespace sofa
>>>>> > > > > > > {
>>>>> > > > > > >
>>>>> > > > > > > namespace component
>>>>> > > > > > > {
>>>>> > > > > > >
>>>>> > > > > > > namespace controller
>>>>> > > > > > > {
>>>>> > > > > > >
>>>>> > > > > > > /* data structure which I want send data to python zmq
>>>>> > > > > > > server */
>>>>> > > > > > > struct instrumentData
>>>>> > > > > > > {
>>>>> > > > > > > typedef sofa::defaulttype::Vec3d Vec3d;
>>>>> > > > > > > Vec3d pos;
>>>>> > > > > > > Quat quat;
>>>>> > > > > > > int btnState;
>>>>> > > > > > > float openInst;
>>>>> > > > > > > bool blnDataReady;
>>>>> > > > > > > };
>>>>> > > > > > >
>>>>> > > > > > > class ZMQComponent : public
>>>>> > > > > > > sofa::core::behavior::BaseController
>>>>> > > > > > > {
>>>>> > > > > > > public:
>>>>> > > > > > > SOFA_CLASS(ZMQComponent,
>>>>> > > > > > > sofa::core::behavior::BaseController);
>>>>> > > > > > >
>>>>> > > > > > > ZMQComponent();
>>>>> > > > > > > virtual ~ZMQComponent();
>>>>> > > > > > > /* Conect to ZMQ external python Server */
>>>>> > > > > > > void setupConnection();
>>>>> > > > > > >
>>>>> > > > > > > /* Send some data memeber instrumentData structure to ZMQ
>>>>> > > > > > > external
>>>>> > > > > > > Server */
>>>>> > > > > > > void instrumentDataSend(instrumentData a);
>>>>> > > > > > >
>>>>> > > > > > > /* initialize function */
>>>>> > > > > > > void init();
>>>>> > > > > > >
>>>>> > > > > > > };
>>>>> > > > > > >
>>>>> > > > > > > } // namespace sofa
>>>>> > > > > > >
>>>>> > > > > > > } // namespace component
>>>>> > > > > > >
>>>>> > > > > > > } // namespace controller
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > The *ZMQComponent.cpp* is:
>>>>> > > > > > >
>>>>> > > > > > > #include <sofa/core/ObjectFactory.h>
>>>>> > > > > > > #include <zmq.hpp>
>>>>> > > > > > > #include <iostream>
>>>>> > > > > > > #include <string>
>>>>> > > > > > > #include "ZMQComponent.h"
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > using namespace std;
>>>>> > > > > > >
>>>>> > > > > > > namespace sofa
>>>>> > > > > > > {
>>>>> > > > > > >
>>>>> > > > > > > namespace component
>>>>> > > > > > > {
>>>>> > > > > > >
>>>>> > > > > > > namespace controller
>>>>> > > > > > > {
>>>>> > > > > > >
>>>>> > > > > > > /* ZMQ Internal Client context and socket */
>>>>> > > > > > > zmq::context_t context(1);
>>>>> > > > > > > zmq::socket_t socket(context, ZMQ_REQ);
>>>>> > > > > > >
>>>>> > > > > > > ZMQComponent::ZMQComponent(){}
>>>>> > > > > > >
>>>>> > > > > > > void ZMQComponent::setupConnection()
>>>>> > > > > > > {
>>>>> > > > > > > cout << "Connecting to python zeroMQ server ..." << endl;
>>>>> > > > > > > socket.connect("tcp://localhost:5555");
>>>>> > > > > > > }
>>>>> > > > > > >
>>>>> > > > > > > void ZMQComponent::instrumentDataSend(instrumentData a)
>>>>> > > > > > > {
>>>>> > > > > > > /* Initialize the data members structure instrumentData */
>>>>> > > > > > > a.pos = sofa::defaulttype::Vec3d(1.0f, 1.0f, 1.0f);
>>>>> > > > > > > a.quat = defaulttype::Quat(1.0f, 1.0f, 4.0f, 1.0f);
>>>>> > > > > > > a.btnState = 5671;
>>>>> > > > > > > a.openInst = 1.0f;
>>>>> > > > > > > a.blnDataReady = false;
>>>>> > > > > > >
>>>>> > > > > > > /* We send the btnState data */
>>>>> > > > > > > zmq::message_t request(10);
>>>>> > > > > > > cout << "The data are: " << a.btnState;
>>>>> > > > > > >
>>>>> > > > > > > /* We ask for the memory address to ge the btnState content
>>>>> > > > > > > and send
>>>>> > > > > > > it. */
>>>>> > > > > > > memcpy(request.data(), &a.btnState, 10);
>>>>> > > > > > > socket.send(request);
>>>>> > > > > > > }
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > /* In the init function we create the objects to setup
>>>>> > > > > > > connection and
>>>>> > > > > > > send data */
>>>>> > > > > > > void ZMQComponent::init()
>>>>> > > > > > > {
>>>>> > > > > > > std::cout << "ZeroMQCommunication::init()" << std::endl;
>>>>> > > > > > > ZMQComponent z;
>>>>> > > > > > > z.setupConnection();
>>>>> > > > > > >
>>>>> > > > > > > instrumentData itemp;
>>>>> > > > > > > z.instrumentDataSend(itemp);
>>>>> > > > > > > }
>>>>> > > > > > >
>>>>> > > > > > > /* Other code related .... */
>>>>> > > > > > > ZMQComponent::~ZMQComponent(){}
>>>>> > > > > > >
>>>>> > > > > > > // int ZeroMqComponentClass =
>>>>> > > > > > > sofa::core::RegisterObject("This
>>>>> > > > > > > component does nothing.").add<ZeroMqComponent>();
>>>>> > > > > > > SOFA_DECL_CLASS(ZMQServerComponent)
>>>>> > > > > > >
>>>>> > > > > > > int ZMQServerComponentClass =
>>>>> > > > > > > sofa::core::RegisterObject("This
>>>>> > > > > > > component create a Socket.").add< ZMQServerComponent >();
>>>>> > > > > > > } // namespace controller
>>>>> > > > > > >
>>>>> > > > > > > } // namespace component
>>>>> > > > > > >
>>>>> > > > > > > } // namespace sofa
>>>>> > > > > > >
>>>>> > > > > > > Then , my python zmq server  which receive the
>>>>> > > > > > > *btnState*  int
>>>>> > > > > > > variable is:
>>>>> > > > > > >
>>>>> > > > > > > import time
>>>>> > > > > > > import zmq
>>>>> > > > > > >
>>>>> > > > > > > context = zmq.Context()
>>>>> > > > > > > socket = context.socket(zmq.REP)
>>>>> > > > > > > socket.bind("tcp://*:5555")
>>>>> > > > > > > print('ZMQ Server listening ... ')
>>>>> > > > > > >
>>>>> > > > > > > while True:
>>>>> > > > > > > # Wait for next request from client
>>>>> > > > > > > message = socket.recv()
>>>>> > > > > > > print("Received message from Sofa: {}".format(message))
>>>>> > > > > > > # print("Received message from c++ %s" % message)
>>>>> > > > > > >
>>>>> > > > > > > # Do some 'work'
>>>>> > > > > > > time.sleep(1)
>>>>> > > > > > >
>>>>> > > > > > > # Send reply back to client
>>>>> > > > > > > # socket.send(b"Hola cliente, muy bien y tu ?")
>>>>> > > > > > > # print('Response sent')
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > The output or the message which arrive to python zmq server
>>>>> > > > > > > is the
>>>>> > > > > > > symbols characters of the btnState:
>>>>> > > > > > >
>>>>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python
>>>>> > > > > > > server.py
>>>>> > > > > > > ZMQ Server listening ...
>>>>> > > > > > > *Received message from Sofa:
>>>>> > > > > > > b"\x00'\x84)\xff\x7f\x00\x00\x98&*"
>>>>> > > > > > >
>>>>> > > > > > > Likely, As I in my python server I am representing like  a
>>>>> > > > > > > string
>>>>> > > > > > > message arrived and I am sending from my c++ client a int
>>>>> > > > > > > btnState
>>>>> > > > > > > data,
>>>>> > > > > > > then I try convert int to string in  the python server
>>>>> > > > > > > side:
>>>>> > > > > > >
>>>>> > > > > > > And I replace this line
>>>>> > > > > > >
>>>>> > > > > > > print("Received message from Sofa: {}".format(message))
>>>>> > > > > > >
>>>>> > > > > > > by thiis line
>>>>> > > > > > >
>>>>> > > > > > > print("Received message from c++ %s" % str(message))
>>>>> > > > > > >
>>>>> > > > > > > But my output is:
>>>>> > > > > > >
>>>>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python
>>>>> > > > > > > server.py
>>>>> > > > > > > ZMQ Server listening ...
>>>>> > > > > > > *Received message from c++
>>>>> > > > > > > b'\x00^n&\xff\x7f\x00\x00\xd8\\'*
>>>>> > > > > > > Traceback (most recent call last):
>>>>> > > > > > >   File "server.py", line 19, in <module>
>>>>> > > > > > >     message = socket.recv()
>>>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 693, in
>>>>> > > > > > > zmq.backend.cython.socket.Socket.recv
>>>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 727, in
>>>>> > > > > > > zmq.backend.cython.socket.Socket.recv
>>>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 150, in
>>>>> > > > > > > zmq.backend.cython.socket._recv_copy
>>>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 145, in
>>>>> > > > > > > zmq.backend.cython.socket._recv_copy
>>>>> > > > > > >   File "zmq/backend/cython/checkrc.pxd", line 25, in
>>>>> > > > > > > zmq.backend.cython.checkrc._check_rc
>>>>> > > > > > > zmq.error.ZMQError: Operation cannot be accomplished in
>>>>> > > > > > > current state
>>>>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗
>>>>> > > > > > >
>>>>> > > > > > > I follow getting the characters symbolss.
>>>>> > > > > > > My question is: How to can I represent the arrived message
>>>>> > > > > > > in my
>>>>> > > > > > > python zmq server to show their content ?
>>>>> > > > > > > According to this behavior, can I assume that the btnState
>>>>> > > > > > > data is
>>>>> > > > > > > sent to python server of anyway?
>>>>> > > > > > >
>>>>> > > > > > > Any support or orientation will be highly appreciated
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > > > _______________________________________________
>>>>> > > > > > > zeromq-dev mailing list
>>>>> > > > > > > zeromq-dev at lists.zeromq.org
>>>>> > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> > > > > > >
>>>>> > > > > >
>>>>> > > > > > _______________________________________________
>>>>> > > > > > zeromq-dev mailing list
>>>>> > > > > > zeromq-dev at lists.zeromq.org
>>>>> > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> > > > > >
>>>>> > > > >
>>>>> > > > > _______________________________________________
>>>>> > > >
>>>>> > > > zeromq-dev mailing list
>>>>> > > > zeromq-dev at lists.zeromq.org
>>>>> > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> > > >
>>>>> > >
>>>>> > > _______________________________________________
>>>>> > > zeromq-dev mailing list
>>>>> > > zeromq-dev at lists.zeromq.org
>>>>> > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> > >
>>>>> > >
>>>>> >
>>>>> > _______________________________________________
>>>>> > zeromq-dev mailing list
>>>>> > zeromq-dev at lists.zeromq.org
>>>>> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>
>>>>> --
>>>>> Kind regards,
>>>>> Luca Boccassi
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>
>>>>>
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>
>>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180130/66089983/attachment.htm>


More information about the zeromq-dev mailing list