[zeromq-dev] Requirements for reliability

Apps, John john.apps at hp.com
Wed Apr 7 16:51:49 CEST 2010


How about a little scenario to get started with? One is probably as good as another, but lottery systems have always intrigued me due to their extremely stringent requirements (defined by the state or government).

Taken very simply, there are just a few types of transaction in a lottery:

1. scan lottery form from customer and submit to backend
1a. process lottery form and store details (on backend)
2. receive confirmation of processing and print ticket (receipt)
3. calculate winner
4. inform winner
5. broadcast information on the lottery (using whatever means)

(1 and 2 above are the same transaction where the sale is in a shop with special hardware for reading the ticket, submitting the bet, printing the receipt. The payment is a separate transaction here as the shopkeeper will not process the ticket until he has money.)

Requirements:

1 and 2 MUST be reliable, i.e., they are in essence a transaction. (Legally, the ticket is as good as whatever the prize happens to be -potentially millions- until such time as the draw has taken place.) 
One could argue that the onus of ensuring consistency lies in the code written on client and server; however, I think that is what the idea of 'reliability' in this context is meant to avoid?

3 is not really relevant here unless the calculation is done on a machine separate from the main backend server. 

4 is similar to 1 and 2 in that the message informing the winner MUST go through to the winner - guaranteed! 
Note: at least as far as the medium by which the winner will be informed, i.e., letter handed to the distribution service or fax sent and confirmed or even an email message (if the legal questions of signature etc. are sorted out).

5 does not need to be reliable in that any old information can be broadcast.

1, 2 and 4 are request/response, I think, with 5 as pub/sub?

One of the big challenges here is the scalability aspect as lottery systems tend to be subject to extremely large fluctuations in demand. One could therefore imagine a step in between 1 and 2, say 1a., where the data is sent to a number of servers for processing/storage in a load-balancing type of environment? This would also require a request/response, this time between backend machines.

Cheers, John
PS: the choice of words is, as always, a problem here. Please consider client and server synonymous with front-end and back-end.

-- John.Apps at hp.com | +491718691813 | http://twitter.com/johnapps --


-----Original Message-----
From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: Wednesday, April 07, 2010 11:51
To: Martin Sustrik
Cc: zeromq-dev at lists.zeromq.org
Subject: Re: [zeromq-dev] Requirements for reliability

On Wed, Apr 7, 2010 at 11:36 AM, Martin Sustrik <sustrik at imatix.com> wrote:

> You'll have to tweak the definition of reliability to get "reliable fanout".

It is also worth considering what failures we want to safeguard
against.  Network failure?  Consumer failure?  Publisher failure?  To
protect against consumer failure, I could imagine a microbroker at
each consumer edge that pulls down messages, and delivers to the
consumer.  Consumer can be slow or fail, it will still get its
messages.

-Pieter
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev



More information about the zeromq-dev mailing list