[zeromq-dev] Formally modelling the publish-subscribe pattern

Brett Viren bv at bnl.gov
Tue Nov 3 16:20:36 CET 2020


Hi John,

John Lång <john.lang at mykolab.com> writes:

>  the large messages are in the order of 12 megabytes

In my limited experience over LANs this is not so overly large to
warrant any special treatment.

> The small messages either contain payload data or the size of the next
> large message

ZeroMQ messages carry their size.  Unless maybe you are trying to enact
some kind of zero-copy you may not need to pre-send an explicit "large
message" size.  (also in my experience zero copy is only useful if
pushing maximum possible throughput, and if that's the goal, there
are many gotcha's)

> The problem is this: What happens if process A sends the small message
> containing the size of the large message to process B, but before B
> has received this message, A already begins sending the large message.

The large message will exist in the sender queue, then its data is
transported, then the message resides in the receiver queue.  From the
two endpoint applications point of view the message is sent atomically.

> Is it possible that the high water mark is
> exceeded and thus the large message gets dropped?

HWM is in units of message count, not bytes.  HWM will be reached when
the number of queued messages get large enough.

Whether subsequent messages are dropped or whether the socket blocks
depends on socket type and socket options.

> How about if A continues sending new unrelated small messages and
> there happens to be enough space for them in the queue? Do these
> messages get through to B?

As long as HWM is not reached there is nothing to impede subsequent
messages.

-Brett.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 849 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20201103/45f37465/attachment.sig>


More information about the zeromq-dev mailing list