[zeromq-dev] High water mark notification for publisher

[Edwin Amsler]

Thanks for the ideas Paul.

My out of band work is just to let late joiners and those clients whose 
disks couldn't write fast enough (they abandon messages destined for 
disk and mark that down). They notify the server of what parts they 
missed, and the server broadcasts again. It's a bit like the Clones 
example in the guide, but the recovery is handled in large batches and 
broadcast out instead of going over a router socket.

So, because of that, I'm less interested in clients not keeping up. If a 
client can't write to disk fast enough, I'm happy to retransmit for them 
once the initial stream is over (this avoids choking the super speedy 
ones). The primary problem here is the interplay between my disks (which 
are unusually fast when cached) and the outgoing network card.

The problem I'm hitting now is that if all clients were infinitely fast, 
I'd still have issues with the network card on the server not being able 
to satisfy the data I'm creating.


On 23/09/2012 7:08 AM, Paul Colomiets wrote:
> Hi Edwin,
>
> On Sun, Sep 23, 2012 at 6:23 AM, Edwin Amsler
> <edwinamsler at thinkboxsoftware.com> wrote:
>> I have a mechanism out of band over TCP that re-requests pieces once the
>> transfer is done, but I'm never actually sure when it's done sending so I
>> just wait 1 minute before re-requesting.
>>
>> If I had some indicator of whether or not the message goes missing, I could
>> re-transmit or throttle back the 500MB/s to what the network is actually
>> able to provide.
>>
> I think you can also have an out of band pub-sub over TCP to send acks
> from clients to server. You need some way to count acks (e.g. clients
> may send presence notification before start of a transfer). But it may
> give you a way to limit bandwidth in more automatic way.
>
> You may also look at ZMQ_RATE and ZMQ_RECOVER_IVL options, they should
> (theoretically) give you reliability without out of band socket to
> request data, however I don't know how well that works in practice.
>