[zeromq-dev] Message-on-expiration hacked into Redis

[Steffen Mueller]

Dear all,

I thought you'd find this entertaining or possibly even useful as a 
fun-with-0MQ example: I hacked support for dispatching 0MQ PUB messages 
from a Redis server whenever a Redis key (string only for now) expires.

It's quite limited functionality, should probably be changed to support 
all Redis types, and likely stands no chance of being incorporated 
upstream, though. It does what I need it to do (extremely high 
read/write load session cache which expires old sessions into a cold 
storage, a fat mysql, from where they can be promoted back into the hot 
cache if necessary).

https://github.com/tsee/redis/tree/msg_on_expire

One detail that I am still wondering about is the type of socket to use:

- There will be many Redis instances on many servers (it's 
single-threaded and did I say there's extremely high read/write load?).
- Eventhough the hot cache is not write-through (yay!), I expect a 
rather high insert load on mysql, so expect to require more than one 
worker that inserts expired sessions into the database.
- All the Redis instances have slaves for resiliency.
- This needs to be able to fail over elegantly without losing too much data.

Thus

=> I will probably switch to PUSH in the expiry hook with multiple 
insert workers PULLing.

=> I could have a worker on the same machine as Redis, one worker per 
Redis instance, but while that would provide more than enough insert 
throughput, it doesn't allow batching of inserts as much for improved 
database load.

=> How can the 0MQ mesh of connections recover if a Redis instance fails 
over to the slave? ==> Redis should reconnect. But of a worker is 
added/removed, the worker needs to reconnect. Who should be binding 
instead of connecting?

What's the general strategy for resilience and failover in such setups 
with 0MQ?

BTW: Did I miss a bit of the 0MQ API that allows checking for dropped 
connections to trigger reconnects?

--Steffen