[zeromq-dev] DEALER to ROUTER and reusing identity.

[Pieter Hintjens]

On Mon, Feb 25, 2013 at 4:23 PM, Trevor Bernard
<trevor.bernard at gmail.com> wrote:

> All I want to do is send work upstream reliably and have the
> ventilator be notified by the sink through an acknowledgement if the
> batch was successful or not so I can take appropriate action.
>
> How would you suggest I go about this?

The ventilator can tell the sink directly how many jobs to expect. If
there's any missing after a timeout, that means a worker died. The
sink would then tell the client "Failed" and the client could decide
to retry, or cancel. This handles the probably most common failure,
which is crashed workers. If you want to handle workers that get into
endless loops but appear to be alive, you would have to add
heartbeating and it all gets more complex. I'd not go there in a first
design.

Trick is to make the simplest possible design you can, break it with
simulated failures, and solve those. The common trap is to over-design
your first architecture.

A reliable pipeline was one of the patterns I'd meant to build but
didn't get around to. Sorry about that.

-Pieter