[zeromq-dev] Formally modelling the publish-subscribe pattern

Brett Viren bv at bnl.gov
Wed Nov 4 15:09:59 CET 2020


Hi again,

John Lång <john.lang at mykolab.com> writes:

> About HWM, do you think it would make sense assuming a HWM of 1 in the
> model? I think HWM 1 would be the worst case scenario where messages
> are most likely to be dropped. HWM 1 would mean that there can be only 
> either one small or one large message in transit at a time. If the
> system satisfies (at least safety) properties in such harsh
> environment, then these properties must also hold when HWM is
> increased, right? Increasing HWM in the model would increase the size
> of the state space exponentially, if I'm not mistaken, and I already
> have many other parameters causing exponential growth. (Last time I
> tried, 2 TB of RAM wasn't enough for simulating even three messages.)

Even with HWM of 1, if the SUB side is faster than the PUB side, and the
network is not a bottleneck, there may not be any dropped messages.

Instead, if I want to induced drops I will add a sufficiently long
sleep() in the SUB loop so that the rate R_pub of message production
outpaces the rate R_sub of consumption.

Assuming constant rates, the difference (R_pub - R_sub) gives the rate
at which the socket queues are filled.  The sum of PUB and SUB HWM
divided by this difference gives the time for the PUB to drop its the
first message.  After another period of that time the SUB will notice
this first lost messages (eg, as a gap in a sequence number carried by
the messages).  You can also calculate and measure things like the
average loss.  Drops should (I think) occur as a run of contiguous
messages with some mean length <N> where actual loss will be over a run
of floor(<N>) or ceil(<N>).  This last is assuming zeromq sockets act
like ideal, simple queues and they may not.

Since the goal is to model this system, it might be useful to make this
sleep() in your code be parameterized by a distribution function so you
can test, eg various constant rate scenarios as well as Gaussian,
Poisson, Exponential, etc, to somewhat model reality.

Anyways, I'm rambling here a bit at the end.  If something gets written
up about your modeling, I'd be interested to read it!

-Brett.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 849 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20201104/be5246ce/attachment.sig>


More information about the zeromq-dev mailing list