[zeromq-dev] publish/subscribe and the "fast producer" problem

[Francesco]

Hi Justin,

2017-03-07 7:39 GMT+01:00 Justin Karneges <justin at karneges.com>:
> On Mon, Mar 6, 2017, at 01:26 AM, Francesco wrote:
>> Hi Justin,
>>
>> >> > My advice
>> >> > is to publish at a fixed rate, for example by sleeping between publishes.
>> >>
>> >> Unfortunately this is not a solution in my case, as the events I
>> >> publish are generated by other incoming events that I cannot control
>> >> and in general I will not know at which rate they arrive.
>> >
>> > Ah, but even if your writes automatically slowed down to the fastest
>> > subscriber, you'd still face this problem. What will you do with
>> > incoming events while you're waiting for the fastest subscriber to
>> > become writable again?
>>
>> In such a case I just need to drop input events.
>
>[...]
>
> It might be handy if there were an alternative to ZMQ_XPUB_NODROP (say
> ZMQ_XPUB_ATLEASTONE) that required at least one subscriber to be
> writable, but not all of them. Then you could send with ZMQ_DONTWAIT,
> and an EAGAIN error would indicate that not even the fastest subscriber
> was writable (you could then discard the message yourself and not retry
> sending it).

Yes, I think this would be really useful not just to me...


> To solve your problem without changing ZeroMQ, you might make a
> subscriber feedback system.
> [...]
I agree, and that's what I'm going to do for the short term. If
someone points me to the right point in the code however I can try to
generate a patch for ZeroMQ adding such a ZMQ_XPUB_ATLEASTONE flag...

thanks,
Francesco