[zeromq-dev] Flow control with router socket

Brett Viren brett.viren at gmail.com
Thu May 14 18:10:35 CEST 2020

I was going to reply similarly.  Once a message is outside a socket,
it's out of ZeroMQ's hands and the responsibility of the application.

This aspect is somewhat fresh in my own learning.  The core issue here
(in my mind) is that the broker (router) severs any direct communication
between producer and consumer.  Thus any back pressure from consumer to
producer must be communicated through the broker application.  It may
sound obvious to say it like this but it took me a bit to internalize
this basic idea.

If it's any help to look at some examples, I've recently implemented a
form of credit-based flow control following the ZeroMQ guide
description.  It adds a runtime choice to use ROUTER/DEALER or
SERVER/CLIENT.  The implementation is somewhat enmeshed in other choices
in my library but it could still serve as a general example.

Actually, I implmented it twice, once in PyZMQ and once in cppzmq.
Barring some bug, the two should inter-operate.





(flow.cpp is kind of a mess due to me still trying to understand
boost.sml which I use for the FSM.  I welcome any critiques if anyone
cares to offer them.)

One caveat, this code is Free to use but I don't consider it released.
I may still end up making drastic changes.

But, to understand (this implementation of) credit-based flow control
this documentation and diagram may be more useful than the code:


And, one last comment: The Python implementation includes a "broker"
that does a little dance in order to hook up the "producer" and the
"consumer" ends in a way that allows each to think they are simple
clients and thus they can reuse the same "flow protocol" code in a
symmetric manner regadless of which role each endpoint plays.  I have
some rambling about how the broker does this dance:


There are likely other, and simpler ways to do this.  I'd be interested
to hear any ideas on that.  The main issue is the broker needs to spawn
a "back end" consumer (producer) to service the "front end" producer
(consumer).  The broker has no a'priori info about its front end clients
and what they may want from a back end so this spawning must be done as
a dymamic reaction to the first message from a "front end".  That and my
desire for symmetric "flow protocol" code leads to this somewhat
contorted dance.


Kevin Sapper <kevinsapper88 at gmail.com> writes:

> Hi Gyorgy,
> back-pressure is something very specific to your application. ZeroMQ itself only implements blocking or dropping depending on the socket types
> you're using.
> To implement a credit-based flow have a look into the guide: http://zguide.zeromq.org/page:all#Transferring-Files
> //Kevin
> Am Do., 14. Mai 2020 um 09:55 Uhr schrieb Gyorgy Szekely <hoditohod at gmail.com>:
>  Hi All,
>  I have a message router that routes messages between different types of producers and consumers. Socket types are router (message
>  router), dealer (producer, consumer). Currently if a consumer is not fast enough messages start to queue up (dealer input queue, router
>  output queue), eventually the router starts to drop messages to avoid blocking.
>  I would like to implement some kind of flow control. With plain TCP I can rely on the built-in flow control: write blocks if consumer is
>  overloaded, and producers can be sanctioned by not reading their socket.
>  With ZMQ router/dealer I can detect  if a consumer is slow by receiving EAGAIN on send, but as far as I understand I can't "slow down" a
>  specific producer, because router socket does fair queuing. So I have to do application layer "stop sending" and "continue" messages and
>  send them to specific producers...
>  Is there any better way to do this? I would rather not reinvent the wheel, TCP already has a sophisticated mechanism for this.
>  Regards,
>     Gyorgy Szekely
>  _______________________________________________
>  zeromq-dev mailing list
>  zeromq-dev at lists.zeromq.org
>  https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 865 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20200514/b7c11c21/attachment.sig>

More information about the zeromq-dev mailing list