[zeromq-dev] A "reliable" PUSH/PULL pattern?

[Randall Nortman]

My apologies if this is somewhere in the guide and I've missed it.
Most of the "reliable" patterns in the guide seem to be about
request/response patterns.  My desired behavior is:

- A producer binds a socket

- A consumer connects to the producer (one consumer, one producer)

- The producer queues up to N messages for a consumer.  As long as the
  queue has not reached N, messages are only removed from the queue
  once the producer is certain that the consumer has received the
  message.  Re-delivery of the same message is acceptable.  Once the
  queue reaches N messages, the producer discards the OLDEST message,
  so that always the most recent N messages are queued.  When the
  consumer comes back, the N most recent messages are transmitted
  (with confirmation of receipt before discarding, subject to queue
  size, as usual).

Based on my reading of the guide, ZMQ doesn't have anything out of the
box that looks like this.  ZMQ's buffers and HWM simply don't work
that way; if anything, they make it slightly harder to reliably do
what I need to do.  The closest I seem to be able to come up with is a
PUSH with the high water mark set to 1 and configured for non-blocking
sends.  If a send would block, then I instead put the message in a
circular queue that I maintain myself.  And I am pretty sure that even
that solution may occasionally drop a message when the consumer
disconnects, because the producer won't notice right away, and
whatever message was in flight at the time will be lost.

Beyond that, it seems like I need to use one of the reliable
request/response patterns, using the responses as confirmation of
receipt (upon which I remove the confirmed message from my queue).  It
would be nice to reduce network traffic by allowing the consumer to
confirm multiple messages in one response (rather than one
confirmation for every message).  That's obviously no longer simple
REQ/REP, but probably something involving more than one socket on each
end.

It seems like the only benefit I get from ZMQ if I want this kind of
behavior is that it handles reconnecting automatically.  I'm kind of
feeling like I might as well use raw UDP, because I'm going to have to
do everything myself anyway.  (I am not knocking ZMQ.  I'm using it
for lots of other things that it's well-suited for.  It just seems
this isn't one of those things.)

Am I missing an easy way to do it?