[zeromq-dev] Formally modelling the publish-subscribe pattern

Brett Viren bv at bnl.gov
Tue Nov 3 17:53:20 CET 2020

John Lång <john.lang at mykolab.com> writes:

> I just like to point out that I'm indeed interested in maximising
> throughput

Over what bandwidth?

Unless you are trying to fill > 20 Gbps, nothing particular special need
be done.  Below that bandwidth, doing anything "special" will needlessly
(imo) complicate your code and/or the application protocol.

> Zero-copy would be ideal for me.

My own tests at 100 Gbps bandwidth found zero copy less important than
other things and these other things bring their own problems.

> How do I get the size of the message?

Try "man zmq_msg_size".  Lots of good reference man pages are provided
with libzmq.

This operates on a msg already in application context.  I don't know of
a way to "peek" into the socket's queue to learn the size of the next
message that would be available if a recv() were called.

> I'm currently using the version of 0MQ I got
> from Debian repositories, which I think is libzmq, though I'm having trouble for
> finding the API documentation for C++.

C++ bindings are developed in projects that are separate from the base
libzmq.  The two leading ones (as I understand the landscape) are the
(separate) projects: zmqpp and cppzmq.  The former is high level,
"batteries included" C++ library while the latter provides "thin", low
level binding as C++ headers.

> I guess documentation for C is good enough.
> I'm using the following function (I copied the declaration straight from <zmq.hpp>)

Hmm, it's not clear to me that you are using libzmq directly.  libzmq
usage would be via zmq.h.  cppzmq provides a "zmq.hpp" so I guess you
are using that.

> for receiving the messages and it asks for the size of the buffer argument:
> inline size_t recv (void *buf_, size_t len_, int flags_ = 0)

This indeed looks like it is from cppzmq.  Note, that particular method
is deprecated.

> This is why I send the size of the large message as a small message,
> so that I can call realloc on buf with len and then pass the pointer
> to 0MQ.

I see.  This is an example of the contortion needed for a zero copy

You can see: https://brettviren.github.io/cppzmq-tour/#org3a79e67 for
two recv() methods which are not deprecated.

The second one may be suitable for "zero copy".  Though, again, that
pattern is imo is more trouble than it's actually worth.

> As of message sizes and HWM, I'm asking these questions because I'm trying to
> consider all possible scenarios instead of just the likely ones. I'm using PUB-SUB
> and it was established earlier in this thread that the publisher will start dropping
> messages in case the HWM is exceeded.

Yes, this is correct. 

> It's good news for me though that HWM is in messages and not bytes. Is
> it possible to check in advance that there will be room for sending at
> least two messages?

Default HWM is 1000 messages.  This can be set to something different
via socket options (separately for send and recv queues).  I don't know
of a way to query it.

I don't think there is a way to ask a PUB (nor XPUB) "are you at HWM for
SUB XYZ?".  The PUB application is blind to subscribers by design.

Note, you CAN send a "multipart" message.  The parts arrive together or
not at all.  You recv() each part separately.

But, again, best to not try to go through these hoops (imo).

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 849 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20201103/d20d5687/attachment.sig>

More information about the zeromq-dev mailing list