[zeromq-dev] Communication between C++ zmq client and Python zmq Server

Thomas Rodgers rodgert at twrodgers.com
Sat Jan 27 17:22:03 CET 2018


Strings are the simplest. If you don’t need to parse the data on the C++
side, just format it as a JSON map on the C++ side, stream it into a strat
team, send the resulting string on the socket and use Python’s support for
parsing JSON on the received strong. For instance.

You can also send structured binary to Python, as I mentioned; the Python
CTypes module (included with Python) will let you define accessors for that
data. As long as the sending and receiving side are the same CPU
architecture you don’t have to worry about endianess conversions.

On Sat, Jan 27, 2018 at 9:13 AM Bernardo Augusto García Loaiza <
botibagl at gmail.com> wrote:

> Hi Luca, this mean, then, with structured string or data is necessary some
> middleware entity like GPB such as Thomas tell us ...
>
>
> Bernardo Augusto García Loaiza
> Ingeniero de Sistemas
> Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT
> http://about.me/bgarcial
>
>
> On Sat, Jan 27, 2018 at 10:07 AM, Luca Boccassi <luca.boccassi at gmail.com>
> wrote:
>
>> Yes you can just send unstructured binary data or strings - Python has
>> native helpers for strings, and CZMQ as well
>>
>> On Fri, 2018-01-26 at 21:27 -0500, Bernardo Augusto García Loaiza
>> wrote:
>> > Thomas, I understand perfectly your explanation. But I want share
>> > with you
>> > another similar case, just for curiosity or to receive some
>> > additional
>> > orientation about it.
>> >
>> > I have a script client C++ ZEROMQ  as a documentation, and I have
>> > script
>> > server python, together based in the documentation zeromq website.
>> > This mean, together are scripts basics
>> >
>> > From C++ client I am sending the string message
>> > <https://github.com/bgarcial/zeroMQ_Client/blob/master/client.cpp#L27
>> > >  and
>> > I have a python server which receive this string message client
>> > <https://github.com/bgarcial/zeroMQ_Client/blob/master/Python/server.
>> > py#L20>
>> > of  a direct way and show me their content
>> >
>> > In this basic sample (documentation zmq based) I am not
>> > using  structures
>> > data  members. Is for this reason that in this case I receive and I
>> > can see
>> > the content of the client message in the zmq server side without have
>> > use
>> > something like Google Protocol Buffer, This mean, is because in this
>> > case I
>> > am not sending any structures data  members?
>> >
>> > I appreciate one more your orientation
>> >
>> >
>> > Bernardo Augusto García Loaiza
>> > Ingeniero de Sistemas
>> > Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT
>> > http://about.me/bgarcial
>> >
>> >
>> > On Fri, Jan 26, 2018 at 7:44 AM, Thomas Rodgers <rodgert at twrodgers.co
>> > m>
>> > wrote:
>> >
>> > > Short answer yes, it’s related to different languages. TCP and
>> > > ZeroMQ only
>> > > deal with transport and framing of messages (respectively) Python
>> > > has no
>> > > idea that that 10 bytes corresponds to a C/C++ struct of a given
>> > > layout
>> > > until you tell it. Different platforms (CPU architectures) have
>> > > potentially
>> > > different representations of fundamental types.
>> > >
>> > > On Thu, Jan 25, 2018 at 4:37 PM Bernardo Augusto García Loaiza <
>> > > botibagl at gmail.com> wrote:
>> > >
>> > > > One more question. Why is necessary the serialization process? It
>> > > > is
>> > > > assumed that zeromq works via TCP and in the TCP protocol, the
>> > > > data does
>> > > > not undergo any transformation? This is related with the fact of
>> > > > that I
>> > > > have different platform/languages?
>> > > >
>> > > > On Thu, Jan 25, 2018 at 4:45 PM Bernardo Augusto García Loaiza <
>> > > > botibagl at gmail.com> wrote:
>> > > >
>> > > > > Hi Thomas,
>> > > > > Thanks for your illustrative response
>> > > > >
>> > > > > I'll look Google Protocol Buffers. My sender is from C++
>> > > > > language and
>> > > > > my reception is Python. What sort of installation recommend me
>> > > > > you?
>> > > > > Binaries or build protocol buffer along my C++ runtime? or o
>> > > > > build protoc
>> > > > > binary from source?
>> > > > >
>> > > > > On Wed, Jan 24, 2018 at 8:36 PM Thomas Rodgers <rodgert at twrodge
>> > > > > rs.com>
>> > > > > wrote:
>> > > > >
>> > > > > > You can have a look at Python’s ctypes module, which will let
>> > > > > > you
>> > > > > > define a ‘struct’ from Python with the same layout as your
>> > > > > > C++ struct.
>> > > > > >
>> > > > > > You can also investigate any number of serialization
>> > > > > > libraries that
>> > > > > > have C++ and Python support, eg ProtoBufs or Thrift, or
>> > > > > > MagPack or whatever.
>> > > > > >
>> > > > > > On Wed, Jan 24, 2018 at 5:26 PM Bernardo Augusto García
>> > > > > > Loaiza <
>> > > > > > botibagl at gmail.com> wrote:
>> > > > > >
>> > > > > > > Hi, ZMQ people.
>> > > > > > > Greetings.
>> > > > > > >
>> > > > > > >
>> > > > > > > I have a  C++ zeromq client process in which I am sending
>> > > > > > > some data
>> > > > > > > members structures
>> > > > > > >
>> > > > > > > *ZMQComponent.h* file
>> > > > > > >
>> > > > > > >
>> > > > > > > #include <zmq.hpp>
>> > > > > > > #include <sofa/defaulttype/VecTypes.h>
>> > > > > > >
>> > > > > > > // To Quat datatype
>> > > > > > > #include <sofa/defaulttype/Quat.h>
>> > > > > > > using sofa::defaulttype::Quat;
>> > > > > > >
>> > > > > > > using std::string;
>> > > > > > >
>> > > > > > > namespace sofa
>> > > > > > > {
>> > > > > > >
>> > > > > > > namespace component
>> > > > > > > {
>> > > > > > >
>> > > > > > > namespace controller
>> > > > > > > {
>> > > > > > >
>> > > > > > > /* data structure which I want send data to python zmq
>> > > > > > > server */
>> > > > > > > struct instrumentData
>> > > > > > > {
>> > > > > > > typedef sofa::defaulttype::Vec3d Vec3d;
>> > > > > > > Vec3d pos;
>> > > > > > > Quat quat;
>> > > > > > > int btnState;
>> > > > > > > float openInst;
>> > > > > > > bool blnDataReady;
>> > > > > > > };
>> > > > > > >
>> > > > > > > class ZMQComponent : public
>> > > > > > > sofa::core::behavior::BaseController
>> > > > > > > {
>> > > > > > > public:
>> > > > > > > SOFA_CLASS(ZMQComponent,
>> > > > > > > sofa::core::behavior::BaseController);
>> > > > > > >
>> > > > > > > ZMQComponent();
>> > > > > > > virtual ~ZMQComponent();
>> > > > > > > /* Conect to ZMQ external python Server */
>> > > > > > > void setupConnection();
>> > > > > > >
>> > > > > > > /* Send some data memeber instrumentData structure to ZMQ
>> > > > > > > external
>> > > > > > > Server */
>> > > > > > > void instrumentDataSend(instrumentData a);
>> > > > > > >
>> > > > > > > /* initialize function */
>> > > > > > > void init();
>> > > > > > >
>> > > > > > > };
>> > > > > > >
>> > > > > > > } // namespace sofa
>> > > > > > >
>> > > > > > > } // namespace component
>> > > > > > >
>> > > > > > > } // namespace controller
>> > > > > > >
>> > > > > > >
>> > > > > > > The *ZMQComponent.cpp* is:
>> > > > > > >
>> > > > > > > #include <sofa/core/ObjectFactory.h>
>> > > > > > > #include <zmq.hpp>
>> > > > > > > #include <iostream>
>> > > > > > > #include <string>
>> > > > > > > #include "ZMQComponent.h"
>> > > > > > >
>> > > > > > >
>> > > > > > > using namespace std;
>> > > > > > >
>> > > > > > > namespace sofa
>> > > > > > > {
>> > > > > > >
>> > > > > > > namespace component
>> > > > > > > {
>> > > > > > >
>> > > > > > > namespace controller
>> > > > > > > {
>> > > > > > >
>> > > > > > > /* ZMQ Internal Client context and socket */
>> > > > > > > zmq::context_t context(1);
>> > > > > > > zmq::socket_t socket(context, ZMQ_REQ);
>> > > > > > >
>> > > > > > > ZMQComponent::ZMQComponent(){}
>> > > > > > >
>> > > > > > > void ZMQComponent::setupConnection()
>> > > > > > > {
>> > > > > > > cout << "Connecting to python zeroMQ server ..." << endl;
>> > > > > > > socket.connect("tcp://localhost:5555");
>> > > > > > > }
>> > > > > > >
>> > > > > > > void ZMQComponent::instrumentDataSend(instrumentData a)
>> > > > > > > {
>> > > > > > > /* Initialize the data members structure instrumentData */
>> > > > > > > a.pos = sofa::defaulttype::Vec3d(1.0f, 1.0f, 1.0f);
>> > > > > > > a.quat = defaulttype::Quat(1.0f, 1.0f, 4.0f, 1.0f);
>> > > > > > > a.btnState = 5671;
>> > > > > > > a.openInst = 1.0f;
>> > > > > > > a.blnDataReady = false;
>> > > > > > >
>> > > > > > > /* We send the btnState data */
>> > > > > > > zmq::message_t request(10);
>> > > > > > > cout << "The data are: " << a.btnState;
>> > > > > > >
>> > > > > > > /* We ask for the memory address to ge the btnState content
>> > > > > > > and send
>> > > > > > > it. */
>> > > > > > > memcpy(request.data(), &a.btnState, 10);
>> > > > > > > socket.send(request);
>> > > > > > > }
>> > > > > > >
>> > > > > > >
>> > > > > > > /* In the init function we create the objects to setup
>> > > > > > > connection and
>> > > > > > > send data */
>> > > > > > > void ZMQComponent::init()
>> > > > > > > {
>> > > > > > > std::cout << "ZeroMQCommunication::init()" << std::endl;
>> > > > > > > ZMQComponent z;
>> > > > > > > z.setupConnection();
>> > > > > > >
>> > > > > > > instrumentData itemp;
>> > > > > > > z.instrumentDataSend(itemp);
>> > > > > > > }
>> > > > > > >
>> > > > > > > /* Other code related .... */
>> > > > > > > ZMQComponent::~ZMQComponent(){}
>> > > > > > >
>> > > > > > > // int ZeroMqComponentClass =
>> > > > > > > sofa::core::RegisterObject("This
>> > > > > > > component does nothing.").add<ZeroMqComponent>();
>> > > > > > > SOFA_DECL_CLASS(ZMQServerComponent)
>> > > > > > >
>> > > > > > > int ZMQServerComponentClass =
>> > > > > > > sofa::core::RegisterObject("This
>> > > > > > > component create a Socket.").add< ZMQServerComponent >();
>> > > > > > > } // namespace controller
>> > > > > > >
>> > > > > > > } // namespace component
>> > > > > > >
>> > > > > > > } // namespace sofa
>> > > > > > >
>> > > > > > > Then , my python zmq server  which receive the
>> > > > > > > *btnState*  int
>> > > > > > > variable is:
>> > > > > > >
>> > > > > > > import time
>> > > > > > > import zmq
>> > > > > > >
>> > > > > > > context = zmq.Context()
>> > > > > > > socket = context.socket(zmq.REP)
>> > > > > > > socket.bind("tcp://*:5555")
>> > > > > > > print('ZMQ Server listening ... ')
>> > > > > > >
>> > > > > > > while True:
>> > > > > > > # Wait for next request from client
>> > > > > > > message = socket.recv()
>> > > > > > > print("Received message from Sofa: {}".format(message))
>> > > > > > > # print("Received message from c++ %s" % message)
>> > > > > > >
>> > > > > > > # Do some 'work'
>> > > > > > > time.sleep(1)
>> > > > > > >
>> > > > > > > # Send reply back to client
>> > > > > > > # socket.send(b"Hola cliente, muy bien y tu ?")
>> > > > > > > # print('Response sent')
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > The output or the message which arrive to python zmq server
>> > > > > > > is the
>> > > > > > > symbols characters of the btnState:
>> > > > > > >
>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python
>> > > > > > > server.py
>> > > > > > > ZMQ Server listening ...
>> > > > > > > *Received message from Sofa:
>> > > > > > > b"\x00'\x84)\xff\x7f\x00\x00\x98&*"
>> > > > > > >
>> > > > > > > Likely, As I in my python server I am representing like  a
>> > > > > > > string
>> > > > > > > message arrived and I am sending from my c++ client a int
>> > > > > > > btnState
>> > > > > > > data,
>> > > > > > > then I try convert int to string in  the python server
>> > > > > > > side:
>> > > > > > >
>> > > > > > > And I replace this line
>> > > > > > >
>> > > > > > > print("Received message from Sofa: {}".format(message))
>> > > > > > >
>> > > > > > > by thiis line
>> > > > > > >
>> > > > > > > print("Received message from c++ %s" % str(message))
>> > > > > > >
>> > > > > > > But my output is:
>> > > > > > >
>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python
>> > > > > > > server.py
>> > > > > > > ZMQ Server listening ...
>> > > > > > > *Received message from c++
>> > > > > > > b'\x00^n&\xff\x7f\x00\x00\xd8\\'*
>> > > > > > > Traceback (most recent call last):
>> > > > > > >   File "server.py", line 19, in <module>
>> > > > > > >     message = socket.recv()
>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 693, in
>> > > > > > > zmq.backend.cython.socket.Socket.recv
>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 727, in
>> > > > > > > zmq.backend.cython.socket.Socket.recv
>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 150, in
>> > > > > > > zmq.backend.cython.socket._recv_copy
>> > > > > > >   File "zmq/backend/cython/socket.pyx", line 145, in
>> > > > > > > zmq.backend.cython.socket._recv_copy
>> > > > > > >   File "zmq/backend/cython/checkrc.pxd", line 25, in
>> > > > > > > zmq.backend.cython.checkrc._check_rc
>> > > > > > > zmq.error.ZMQError: Operation cannot be accomplished in
>> > > > > > > current state
>> > > > > > > (cnvss_test) ➜  Python git:(ZMQCommunication) ✗
>> > > > > > >
>> > > > > > > I follow getting the characters symbolss.
>> > > > > > > My question is: How to can I represent the arrived message
>> > > > > > > in my
>> > > > > > > python zmq server to show their content ?
>> > > > > > > According to this behavior, can I assume that the btnState
>> > > > > > > data is
>> > > > > > > sent to python server of anyway?
>> > > > > > >
>> > > > > > > Any support or orientation will be highly appreciated
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > _______________________________________________
>> > > > > > > zeromq-dev mailing list
>> > > > > > > zeromq-dev at lists.zeromq.org
>> > > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> > > > > > >
>> > > > > >
>> > > > > > _______________________________________________
>> > > > > > zeromq-dev mailing list
>> > > > > > zeromq-dev at lists.zeromq.org
>> > > > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> > > > > >
>> > > > >
>> > > > > _______________________________________________
>> > > >
>> > > > zeromq-dev mailing list
>> > > > zeromq-dev at lists.zeromq.org
>> > > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> > > >
>> > >
>> > > _______________________________________________
>> > > zeromq-dev mailing list
>> > > zeromq-dev at lists.zeromq.org
>> > > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> > >
>> > >
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> --
>> Kind regards,
>> Luca Boccassi
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180127/550cc4da/attachment.htm>


More information about the zeromq-dev mailing list