[zeromq-dev] Indefinite Retention and Durable Sockets

[Gregory Szorc]

The zguide roughly states in Chapter 4 [1]:

> [For durable sockets connected to a ROUTER] 0MQ will never throw away messages to a server it's seen, even if the server does go away.

First, the zmq_socket() man page doesn't seem to mention this property
explicitly. The documentation in Git for 2.1 even seems to be
contradictory [2]:

> any messages routed to a non-existent peer... shall also be dropped.

So what is the actual behavior? Does it vary between library versions?
(I'll happily submit a docs pull request once I discover truth on this,
if you want.)

If the "never throws away" description is accurate, it seems very
troublesome. (If not, you can probably ignore the rest of this post.)
How do you handle clients that establish durable sockets, send a message
that will incur a response, then disconnect before that response is
delivered and never reconnect to receive the queued message? It seems
this behavior will effectively cause a memory leak and there is no solid
mitigation strategy.

HWM won't work, because it is on a per-peer basis and clients could
connect with different identities (a malicious peer could do this in a
loop to kill a remote agent).

It seems the best strategy today is to not use durable sockets. But,
what if I have no/little control over remote client implementations and
one of these uses durable sockets without realizing the consequences?
All it takes is one client disconnect at the right time and we have
orphaned messages and a memory leak.

Has there been any consideration to any of the following to help
mitigate this problem:

1) A socket option that disallows peers from having explicit identities.
It is stated in the guide that durable sockets were probably a bad idea.
Can we have a socket option to enforce this behavior?

2) A socket option that causes sends to disconnected durable peers to be
(transparently?) dropped on the floor.

3) A socket API to clear messages waiting to be delivered to
disconnected peers.

4) A TTL on messages waiting to be delivered to a disconnected peer.

Keep up the great work!

Gregory Szorc
gregory.szorc at gmail.com

[1] https://github.com/imatix/zguide/blob/master/chapter4.txt#L1165
[2] https://github.com/zeromq/zeromq2-1/blob/master/doc/zmq_socket.txt#L159