[zeromq-dev] DEALER to ROUTER and reusing identity.
ph at imatix.com
Mon Feb 25 13:52:12 CET 2013
On Mon, Feb 25, 2013 at 1:24 PM, Trevor Bernard
<trevor.bernard at gmail.com> wrote:
> The pattern I'm trying to implement is a reliable pipeline. I want to
> be able to guarantee that what I send from the front endpoint won't be
> lost on it's way to the back endpoint.
OK, the simplest design is to assume that failure is possible in
workers due to them running application code, whereas ventilator and
sink are robust. Then, you detect failure by having a loss in the
sink, and you recover from failure by resubmitting the whole batch. If
it fails more than once, you flag it as "toxic" and stop retrying.
If you want to handle failed sinks or ventilators, you can use
redundancy and simply choose a sink/ventilator at start up, since
these are stateless.
You can do all this with PUSH/PULL I think.
Clearly you're trying something more sophisticated so my guess is
either we don't mean the same thing with "reliable pipeline", or
you're building a too-complex solution.
My advice on reliability is to identify real failures and solve those
in order of probability.
More information about the zeromq-dev