[zeromq-dev] publish/subscribe and the "fast producer" problem

[Francesco]

Hi Justin,

>> > My advice
>> > is to publish at a fixed rate, for example by sleeping between publishes.
>>
>> Unfortunately this is not a solution in my case, as the events I
>> publish are generated by other incoming events that I cannot control
>> and in general I will not know at which rate they arrive.
>
> Ah, but even if your writes automatically slowed down to the fastest
> subscriber, you'd still face this problem. What will you do with
> incoming events while you're waiting for the fastest subscriber to
> become writable again?

In such a case I just need to drop input events.
My idea is however to collect some stat to show to a human that, in
this way, will be able to understand where the bottleneck is: maybe
it's the network between sub-pub (in such case: go buy a faster
switch) or maybe it's the subs being too slow (in that case, spin off
more subscribers - in my case they are VMs).

The advantage of bounding the pub to the FASTEST subscriber is, as
mentioned in my prev emails, that I will drop events at the very input
of my system (i.e., at the input of the subscriber) ONLY if the system
slows down up to a point where even the FASTEST subscriber cannot keep
up with the event rate... does it makes sense? :)

Francesco