[zeromq-dev] How to implement lossless pub-sub with ZeroMQ?

[Antti Karanta]

I was wondering about loss of messages I observed with PUB-SUB although 
I had set send and recieve high water marks to 0. I reread the 
documentation and sure enough, PUB-SUB is designed to lose messages in 
some circumstances:

"The ZeroMQ pub-sub pattern will lose messages arbitrarily when a 
subscriber is connecting, when a network failure occurs, or just if the 
subscriber or network can't keep up with the publisher."
http://zguide.zeromq.org/php:chapter5#Pros-and-Cons-of-Pub-Sub

Thus PUB-SUB is not really suited for my use case as loss of messages, 
while not fatal, is highly inconvenient. Detecting loss of messages will 
force the client to disconnect, after which a user must reopen his work 
(= load a lot of data) and this is annoying to say the least. The 
detection of message loss is done simply by numbering the messages.


Are there any suggestions on how to implement "lossless" (to a 
reasonable extent, allowing at least timeouts, i.e. a message needs not 
be delivered if the recipient can't be reached for given amount of time) 
multicast with ZeroMQ?
I've been thinking of implementing this with dealer and router sockets 
but am not sure whether this is the best way.

Here are some further details of my scenario. If more are needed to 
provide an answer to my question I will be happy to provide it.

   * the basic idea that there are a number of clients editing the same 
data with a server relaying the changes each of them makes to the others 
(cf. a document in google docs) and saving the changes to a persistent 
storage
   * the clients request the present state of the data they are 
interested in (implemented with async server pattern using dealer-router 
sockets) and simultaneously (actually before) subscribe to changes to 
that data using pub-sub and a filter specific to that set of data
     ** appropriate care is taken to tell which change messages are 
already included in the state received as a reply to the request and 
they are discarded
   * the client must not miss any of the change messages in the 
broadcast or it will be in a corrupt state (its local data not 
reflecting what the data looks like at the moment)
   * if a client restarts it does not require the change messages 
broadcast in between
   * clients will detect if server is restarted and also in this case 
force reloading of the data
   * the amount of data broadcast is not too big compared to the amount 
of available memory
   * the messages are quickly read on the client and passed to other 
threads to be dealt with (i.e. they won't stay in ZeroMQ's buffers very 
long)
   * the server broadcasts heartbeats roughly every 1500ms

environment: cljzmq on the server, clrzmr on the client, zmq version 
4.0.4 (x64) on both