[zeromq-dev] DEALER-ROUTER question

Thomas Rodgers rodgert at twrodgers.com
Thu Sep 25 16:48:48 CEST 2014


Sigh, no good deed goes unpunished...

To your original question -

As covered in the zguide, ZeroMQ is already managing a background thread on
your behalf that is 'pumping' the underlying transport's socket and pulling
messages/sending messages as fast as it can (using epoll, select, IOCP,
etc. as appropriate for the platform).

It is your application's responsibility to determine how best to integrate
that with the application's run-loop on the sending/receiving end of the
ZeroMQ socket. Typically this would done with zmq_poll
<http://api.zeromq.org/4-0:zmq-poll> and there's a zguide sample to get you
started here <http://zguide.zeromq.org/cpp:mspoller>. Polling with a
timeout of 0 in a while(1) loop will essentially 'burn' the calling thread
for the purposes of notifying the caller of which sockets have at least one
message ready to be received (ZMQ_POLLIN) and/or at least one message can
be sent (ZMQ_POLLOUT).

How to best integrate this into your application, is highly dependent on
your application, it's latency requirements, etc.

To the rest -

Pieter has certainly done more work on libzmq/czmq, it's documentation and
ZeroMQ advocacy than any other individual.  Pieter continues to do
substantial ongoing development work on ZeroMQ, hardly consistent with
usual connotations attached to the term "figurehead".

I have to admit I'm a bit baffled at the assertion about hwserver code in
the zguide not demonstrating how to pull out the contents of the payload.
It is precisely the same as putting the payload *INTO* the message, with
msg.data() being the source rather than the destination argument to memcpy,
and you needing to provide an appropriate destination (char array,
std::array<char>, std::vector<char>) and calling msg.size() to get the
actual message size.

The zguide is not intended to be a guide on basic usage of standard C/C++,
their common idioms and standard libraries.

MSDN has complete documentation on the use of memcpy at -

http://msdn.microsoft.com/en-us/library/dswaw1wk.aspx

Nor is the zguide intended to be a more verbose replacement for the ZeroMQ
reference manual -

http://api.zeromq.org/

You actually do need to use both documents.

If you are basing your code on zmq.hpp, the zguide is in no way intended to
replace reading the source -

https://github.com/zeromq/cppzmq/blob/master/zmq.hpp

It is a very thin wrapper over the libzmq API, which is the only core
documentation requirement.

If you want to serialize objects and send them over ZMQ, you should be
looking at an existing framework (MFC has one, so does Boost
<http://www.boost.org/doc/libs/1_56_0/libs/serialization/doc/index.html>).
Both support serialization to an arbitrary byte buffer, of which
zmq::message_t is, but it is well beyond the scope of the zguide to tell
you how, for instance, you might implement the Boost Serialization archive
concept (or *anything* to do with MFC) on top of zmq::message_t.

I spent a significant chunk of the last 25 years as a C++ developer on
Microsoft's platforms and using Microsoft's development tools. In my
experience it has been exceedingly rare to find open source projects which
begin life on *nix that magically 'just work' on the Windows platform. When
I needed a particular bit of open source to work on Windows, I did the work
fix the issues, recognizing that Windows was a decided non-priority for the
developers of the project in question. That ZeroMQ attempts to provide
actual builds and ports for Windows is a Good Thing, but it requires the
people who actually use that platform regularly (no longer one of those
people, sorry) to contribute to keeping those build targets current.



On Thu, Sep 25, 2014 at 8:08 AM, Riskybiz <riskybizlive at live.com> wrote:

> Dear Pieter,
>
> As a figurehead of the zeromq project I think you should know that the
> zguide and its examples, whilst a worthy effort, is actually a barrier to
> the progress of a learner.  As a newcomer to networking my interactions and
> attempts to learn and use zeromq over the past year has been an unyielding
> source of issues to be overcome.  Anyone else would probably have given up,
> but I recognise the benefits of the superb technology offered by zeromq.
>
> Part of the problem is perhaps that the zguide has been written by experts,
> where seemingly minor details are glossed over.  For example in the
> hwserver
> and hwclient code it is not demonstrated how actually to read a message
> payload from the socket and extract the received message string.  It cheats
> and prints; printf ("Received World %d\n", request_nbr);  When someone
> tries
> this early example the first thing they will experiment with is:  How can I
> customise the message payload and get my own message sent and received?
> They will be disappointed.  They might read on try to figure it out and be
> baffled by the multiple language bindings, apis and helper files and left
> wondering; what actually needs to be written to make zeromq work?  What is
> the core underlying zeromq commands which need to be called?  What needs to
> be installed? How do I do that? Where do I find the downloads?
>
> I encountered a situation last year when I needed to use the common
> technique of serialization to pass custom C++ objects over zeromq sockets.
> While the zguide alludes to the possibility there was nothing to offer
> assistance in the practical implementation.  It took some considerable
> time,
> weeks, to sort this out into a functioning prototype.  Where a simple
> functional guide example could have saved time and questions; not just for
> me but also for any number of other users.
>
> I find that the zguide code examples are in themselves problematic.  The
> example code is very sparsely commented.  Every line which does something
> non-trivial or zeromq specific should be explained.   I have found examples
> to be zeromq version specific, operating system specific and requiring
> modifications to work on Windows.  All of these factors just consume time
> in
> endeavouring to make them work or debug them with limited understanding of
> what is actually supposed to be happening.  This causes questions and
> frustrations.
>
> CZMQ was recommended as the api to use with 'reference' C language code
> examples, I lost more time trying to compile CZMQ before realising the
> practical impossibility of this on Windows despite alluringly providing
> Visual Studio project files.  The lack of working installation instructions
> was also a barrier.  Then afterwards I discover that ROUTER sockets in
> updated zeromq versions no longer use UUID identities thus anyway
> invalidating the code example I was endeavouring to get working.  More lost
> time, more questions.   No progress.
>
> My suggestion is that if you want fewer basic questions asked in the
> community then please take time to revisit the zguide, its examples and
> necessary zeromq code resources and make it such that people can easily
> find
> the resources they need, confidently learn and demonstrate the examples and
> functionality for themselves; without enduring endless technical barriers
> and frustrations.
>
> I hope you will view my feedback as constructive.
>
> All that said; which part of the guide should I read to solve my
> DEALER-ROUTER issue, can I be confident that any associated zguide example
> will actually work and help me to further understand how to solve my own
> zeromq issues in the future?
>
>
> Riskybiz.
>
>
> Message: 29
> Date: Thu, 25 Sep 2014 11:29:01 +0200
> From: Pieter Hintjens <ph at imatix.com>
> Subject: Re: [zeromq-dev] DEALER-ROUTER question.
> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
> Message-ID:
>         <CADL5_sjhfSqq_YDJq59pindtY3t15dUpiJVO3wr=
> qEHn+tYeJA at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Please read the Guide and work through the examples, it has lots of
> explanation and examples of how to do this kind of thing.
>
> On Thu, Sep 25, 2014 at 10:36 AM, Riskybiz <riskybizlive at live.com> wrote:
> > A quick question for the zeromq experts if I may?
> >
> >
> >
> > Given a DEALER to ROUTER zeromq connection which is able to operate
> > asynchronously; what is the best way to coordinate inbound and outbound
> > message handling such that the DEALER socket is not blocked and an
> > application is able to react and send at a given moment but also to
> promptly
> > receive?
> >
> >
> >
> > Thanks,
> >
> >
> >
> > Riskybiz.
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
>
>
> ------------------------------
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> End of zeromq-dev Digest, Vol 81, Issue 25
> ******************************************
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140925/8b1c5f85/attachment.htm>


More information about the zeromq-dev mailing list