[zeromq-dev] Communication between C++ zmq client and Python zmq Server

Thomas Rodgers rodgert at twrodgers.com
Sat Jan 27 17:24:48 CET 2018


“strat team” was supposed to be std::stringstream (Thanks for nothing
autocorrect!)

http://en.cppreference.com/w/cpp/io/basic_stringstream

On Sat, Jan 27, 2018 at 10:22 AM Thomas Rodgers <rodgert at twrodgers.com>
wrote:

> Strings are the simplest. If you don’t need to parse the data on the C++
> side, just format it as a JSON map on the C++ side, stream it into a strat
> team, send the resulting string on the socket and use Python’s support for
> parsing JSON on the received strong. For instance.
>
> You can also send structured binary to Python, as I mentioned; the Python
> CTypes module (included with Python) will let you define accessors for that
> data. As long as the sending and receiving side are the same CPU
> architecture you don’t have to worry about endianess conversions.
>
> On Sat, Jan 27, 2018 at 9:13 AM Bernardo Augusto García Loaiza <
> botibagl at gmail.com> wrote:
>
>> Hi Luca, this mean, then, with structured string or data is necessary
>> some middleware entity like GPB such as Thomas tell us ...
>>
>>
>> Bernardo Augusto García Loaiza
>> Ingeniero de Sistemas
>> Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT
>> http://about.me/bgarcial
>>
>>
>> On Sat, Jan 27, 2018 at 10:07 AM, Luca Boccassi <luca.boccassi at gmail.com>
>> wrote:
>>
>>> Yes you can just send unstructured binary data or strings - Python has
>>> native helpers for strings, and CZMQ as well
>>>
>>> On Fri, 2018-01-26 at 21:27 -0500, Bernardo Augusto García Loaiza
>>> wrote:
>>> > Thomas, I understand perfectly your explanation. But I want share
>>> > with you
>>> > another similar case, just for curiosity or to receive some
>>> > additional
>>> > orientation about it.
>>> >
>>> > I have a script client C++ ZEROMQ  as a documentation, and I have
>>> > script
>>> > server python, together based in the documentation zeromq website.
>>> > This mean, together are scripts basics
>>> >
>>> > From C++ client I am sending the string message
>>> > <https://github.com/bgarcial/zeroMQ_Client/blob/master/client.cpp#L27
>>> > >  and
>>> > I have a python server which receive this string message client
>>> > <https://github.com/bgarcial/zeroMQ_Client/blob/master/Python/server.
>>> > py#L20>
>>> > of  a direct way and show me their content
>>> >
>>> > In this basic sample (documentation zmq based) I am not
>>> > using  structures
>>> > data  members. Is for this reason that in this case I receive and I
>>> > can see
>>> > the content of the client message in the zmq server side without have
>>> > use
>>> > something like Google Protocol Buffer, This mean, is because in this
>>> > case I
>>> > am not sending any structures data  members?
>>> >
>>> > I appreciate one more your orientation
>>> >
>>> >
>>> > Bernardo Augusto García Loaiza
>>> > Ingeniero de Sistemas
>>> > Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT
>>> > http://about.me/bgarcial
>>> >
>>> >
>>> > On Fri, Jan 26, 2018 at 7:44 AM, Thomas Rodgers <rodgert at twrodgers.co
>>> > m>
>>> > wrote:
>>> >
>>> > > Short answer yes, it’s related to different languages. TCP and
>>> > > ZeroMQ only
>>> > > deal with transport and framing of messages (respectively) Python
>>> > > has no
>>> > > idea that that 10 bytes corresponds to a C/C++ struct of a given
>>> > > layout
>>> > > until you tell it. Different platforms (CPU architectures) have
>>> > > potentially
>>> > > different representations of fundamental types.
>>> > >
>>> > > On Thu, Jan 25, 2018 at 4:37 PM Bernardo Augusto García Loaiza <
>>> > > botibagl at gmail.com> wrote:
>>> > >
>>> > > > One more question. Why is necessary the serialization process? It
>>> > > > is
>>> > > > assumed that zeromq works via TCP and in the TCP protocol, the
>>> > > > data does
>>> > > > not undergo any transformation? This is related with the fact of
>>> > > > that I
>>> > > > have different platform/languages?
>>> > > >
>>> > > > On Thu, Jan 25, 2018 at 4:45 PM Bernardo Augusto García Loaiza <
>>> > > > botibagl at gmail.com> wrote:
>>> > > >
>>> > > > > Hi Thomas,
>>> > > > > Thanks for your illustrative response
>>> > > > >
>>> > > > > I'll look Google Protocol Buffers. My sender is from C++
>>> > > > > language and
>>> > > > > my reception is Python. What sort of installation recommend me
>>> > > > > you?
>>> > > > > Binaries or build protocol buffer along my C++ runtime? or o
>>> > > > > build protoc
>>> > > > > binary from source?
>>> > > > >
>>> > > > > On Wed, Jan 24, 2018 at 8:36 PM Thomas Rodgers <rodgert at twrodge
>>> > > > > rs.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > You can have a look at Python’s ctypes module, which will let
>>> > > > > > you
>>> > > > > > define a ‘struct’ from Python with the same layout as your
>>> > > > > > C++ struct.
>>> > > > > >
>>> > > > > > You can also investigate any number of serialization
>>> > > > > > libraries that
>>> > > > > > have C++ and Python support, eg ProtoBufs or Thrift, or
>>> > > > > > MagPack or whatever.
>>> > > > > >
>>> > > > > > On Wed, Jan 24, 2018 at 5:26 PM Bernardo Augusto García
>>> > > > > > Loaiza <
>>> > > > > > botibagl at gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Hi, ZMQ people.
>>> > > > > > > Greetings.
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > I have a  C++ zeromq client process in which I am sending
>>> > > > > > > some data
>>> > > > > > > members structures
>>> > > > > > >
>>> > > > > > > *ZMQComponent.h* file
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > #include <zmq.hpp>
>>> > > > > > > #include <sofa/defaulttype/VecTypes.h>
>>> > > > > > >
>>> > > > > > > // To Quat datatype
>>> > > > > > > #include <sofa/defaulttype/Quat.h>
>>> > > > > > > using sofa::defaulttype::Quat;
>>> > > > > > >
>>> > > > > > > using std::string;
>>> > > > > > >
>>> > > > > > > namespace sofa
>>> > > > > > > {
>>> > > > > > >
>>> > > > > > > namespace component
>>> > > > > > > {
>>> > > > > > >
>>> > > > > > > namespace controller
>>> > > > > > > {
>>> > > > > > >
>>> > > > > > > /* data structure which I want send data to python zmq
>>> > > > > > > server */
>>> > > > > > > struct instrumentData
>>> > > > > > > {
>>> > > > > > > typedef sofa::defaulttype::Vec3d Vec3d;
>>> > > > > > > Vec3d pos;
>>> > > > > > > Quat quat;
>>> > > > > > > int btnState;
>>> > > > > > > float openInst;
>>> > > > > > > bool blnDataReady;
>>> > > > > > > };
>>> > > > > > >
>>> > > > > > > class ZMQComponent : public
>>> > > > > > > sofa::core::behavior::BaseController
>>> > > > > > > {
>>> > > > > > > public:
>>> > > > > > > SOFA_CLASS(ZMQComponent,
>>> > > > > > > sofa::core::behavior::BaseController);
>>> > > > > > >
>>> > > > > > > ZMQComponent();
>>> > > > > > > virtual ~ZMQComponent();
>>> > > > > > > /* Conect to ZMQ external python Server */
>>> > > > > > > void setupConnection();
>>> > > > > > >
>>> > > > > > > /* Send some data memeber instrumentData structure to ZMQ
>>> > > > > > > external
>>> > > > > > > Server */
>>> > > > > > > void instrumentDataSend(instrumentData a);
>>> > > > > > >
>>> > > > > > > /* initialize function */
>>> > > > > > > void init();
>>> > > > > > >
>>> > > > > > > };
>>> > > > > > >
>>> > > > > > > } // namespace sofa
>>> > > > > > >
>>> > > > > > > } // namespace component
>>> > > > > > >
>>> > > > > > > } // namespace controller
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > The *ZMQComponent.cpp* is:
>>> > > > > > >
>>> > > > > > > #include <sofa/core/ObjectFactory.h>
>>> > > > > > > #include <zmq.hpp>
>>> > > > > > > #include <iostream>
>>> > > > > > > #include <string>
>>> > > > > > > #include "ZMQComponent.h"
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > using namespace std;
>>> > > > > > >
>>> > > > > > > namespace sofa
>>> > > > > > > {
>>> > > > > > >
>>> > > > > > > namespace component
>>> > > > > > > {
>>> > > > > > >
>>> > > > > > > namespace controller
>>> > > > > > > {
>>> > > > > > >
>>> > > > > > > /* ZMQ Internal Client context and socket */
>>> > > > > > > zmq::context_t context(1);
>>> > > > > > > zmq::socket_t socket(context, ZMQ_REQ);
>>> > > > > > >
>>> > > > > > > ZMQComponent::ZMQComponent(){}
>>> > > > > > >
>>> > > > > > > void ZMQComponent::setupConnection()
>>> > > > > > > {
>>> > > > > > > cout << "Connecting to python zeroMQ server ..." << endl;
>>> > > > > > > socket.connect("tcp://localhost:5555");
>>> > > > > > > }
>>> > > > > > >
>>> > > > > > > void ZMQComponent::instrumentDataSend(instrumentData a)
>>> > > > > > > {
>>> > > > > > > /* Initialize the data members structure instrumentData */
>>> > > > > > > a.pos = sofa::defaulttype::Vec3d(1.0f, 1.0f, 1.0f);
>>> > > > > > > a.quat = defaulttype::Quat(1.0f, 1.0f, 4.0f, 1.0f);
>>> > > > > > > a.btnState = 5671;
>>> > > > > > > a.openInst = 1.0f;
>>> > > > > > > a.blnDataReady = false;
>>> > > > > > >
>>> > > > > > > /* We send the btnState data */
>>> > > > > > > zmq::message_t request(10);
>>> > > > > > > cout << "The data are: " << a.btnState;
>>> > > > > > >
>>> > > > > > > /* We ask for the memory address to ge the btnState content
>>> > > > > > > and send
>>> > > > > > > it. */
>>> > > > > > > memcpy(request.data(), &a.btnState, 10);
>>> > > > > > > socket.send(request);
>>> > > > > > > }
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > /* In the init function we create the objects to setup
>>> > > > > > > connection and
>>> > > > > > > send data */
>>> > > > > > > void ZMQComponent::init()
>>> > > > > > > {
>>> > > > > > > std::cout << "ZeroMQCommunication::init()" << std::endl;
>>> > > > > > > ZMQComponent z;
>>> > > > > > > z.setupConnection();
>>> > > > > > >
>>> > > > > > > instrumentData itemp;
>>> > > > > > > z.instrumentDataSend(itemp);
>>> > > > > > > }
>>> > > > > > >
>>> > > > > > > /* Other code related .... */
>>> > > > > > > ZMQComponent::~ZMQComponent(){}
>>> > > > > > >
>>> > > > > > > // int ZeroMqComponentClass =
>>> > > > > > > sofa::core::RegisterObject("This
>>> > > > > > > component does nothing.").add<ZeroMqComponent>();
>>> > > > > > > SOFA_DECL_CLASS(ZMQServerComponent)
>>> > > > > > >
>>> > > > > > > int ZMQServerComponentClass =
>>> > > > > > > sofa::core::RegisterObject("This
>>> > > > > > > component create a Socket.").add< ZMQServerComponent >();
>>> > > > > > > } // namespace controller
>>> > > > > > >
>>> > > > > > > } // namespace component
>>> > > > > > >
>>> > > > > > > } // namespace sofa
>>> > > > > > >
>>> > > > > > > Then , my python zmq server  which receive the
>>> > > > > > > *btnState*  int
>>> > > > > > > variable is:
>>> > > > > > >
>>> > > > > > > import time
>>> > > > > > > import zmq
>>> > > > > > >
>>> > > > > > > context = zmq.Context()
>>> > > > > > > socket = context.socket(zmq.REP)
>>> > > > > > > socket.bind("tcp://*:5555")
>>> > > > > > > print('ZMQ Server listening ... ')
>>> > > > > > >
>>> > > > > > > while True:
>>> > > > > > > # Wait for next request from client
>>> > > > > > > message = socket.recv()
>>> > > > > > > print("Received message from Sofa: {}".format(message))
>>> > > > > > > # print("Received message from c++ %s" % message)
>>> > > > > > >
>>> > > > > > > # Do some 'work'
>>> > > > > > > time.sleep(1)
>>> > > > > > >
>>> > > > > > > # Send reply back to client
>>> > > > > > > # socket.send(b"Hola cliente, muy bien y tu ?")
>>> > > > > > > # print('Response sent')
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > The output or the message which arrive to python zmq server
>>> > > > > > > is the
>>> > > > > > > symbols characters of the btnState:
>>> > > > > > >
>>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python
>>> > > > > > > server.py
>>> > > > > > > ZMQ Server listening ...
>>> > > > > > > *Received message from Sofa:
>>> > > > > > > b"\x00'\x84)\xff\x7f\x00\x00\x98&*"
>>> > > > > > >
>>> > > > > > > Likely, As I in my python server I am representing like  a
>>> > > > > > > string
>>> > > > > > > message arrived and I am sending from my c++ client a int
>>> > > > > > > btnState
>>> > > > > > > data,
>>> > > > > > > then I try convert int to string in  the python server
>>> > > > > > > side:
>>> > > > > > >
>>> > > > > > > And I replace this line
>>> > > > > > >
>>> > > > > > > print("Received message from Sofa: {}".format(message))
>>> > > > > > >
>>> > > > > > > by thiis line
>>> > > > > > >
>>> > > > > > > print("Received message from c++ %s" % str(message))
>>> > > > > > >
>>> > > > > > > But my output is:
>>> > > > > > >
>>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python
>>> > > > > > > server.py
>>> > > > > > > ZMQ Server listening ...
>>> > > > > > > *Received message from c++
>>> > > > > > > b'\x00^n&\xff\x7f\x00\x00\xd8\\'*
>>> > > > > > > Traceback (most recent call last):
>>> > > > > > >   File "server.py", line 19, in <module>
>>> > > > > > >     message = socket.recv()
>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 693, in
>>> > > > > > > zmq.backend.cython.socket.Socket.recv
>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 727, in
>>> > > > > > > zmq.backend.cython.socket.Socket.recv
>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 150, in
>>> > > > > > > zmq.backend.cython.socket._recv_copy
>>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 145, in
>>> > > > > > > zmq.backend.cython.socket._recv_copy
>>> > > > > > >   File "zmq/backend/cython/checkrc.pxd", line 25, in
>>> > > > > > > zmq.backend.cython.checkrc._check_rc
>>> > > > > > > zmq.error.ZMQError: Operation cannot be accomplished in
>>> > > > > > > current state
>>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗
>>> > > > > > >
>>> > > > > > > I follow getting the characters symbolss.
>>> > > > > > > My question is: How to can I represent the arrived message
>>> > > > > > > in my
>>> > > > > > > python zmq server to show their content ?
>>> > > > > > > According to this behavior, can I assume that the btnState
>>> > > > > > > data is
>>> > > > > > > sent to python server of anyway?
>>> > > > > > >
>>> > > > > > > Any support or orientation will be highly appreciated
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > _______________________________________________
>>> > > > > > > zeromq-dev mailing list
>>> > > > > > > zeromq-dev at lists.zeromq.org
>>> > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> > > > > > >
>>> > > > > >
>>> > > > > > _______________________________________________
>>> > > > > > zeromq-dev mailing list
>>> > > > > > zeromq-dev at lists.zeromq.org
>>> > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> > > > > >
>>> > > > >
>>> > > > > _______________________________________________
>>> > > >
>>> > > > zeromq-dev mailing list
>>> > > > zeromq-dev at lists.zeromq.org
>>> > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> > > >
>>> > >
>>> > > _______________________________________________
>>> > > zeromq-dev mailing list
>>> > > zeromq-dev at lists.zeromq.org
>>> > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> > >
>>> > >
>>> >
>>> > _______________________________________________
>>> > zeromq-dev mailing list
>>> > zeromq-dev at lists.zeromq.org
>>> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>> --
>>> Kind regards,
>>> Luca Boccassi
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180127/e060b473/attachment.htm>


More information about the zeromq-dev mailing list