[zeromq-dev] Distributed Event Bus

Tom Cocagne tom.cocagne at gmail.com
Thu Feb 21 17:02:59 CET 2013


One thing to note with the Paxos option, if you decide to go that
route, is that the number of participants in the decision making
process doesn't scale all that well. Typically, a sub-set of nodes (<
30 or so) is used as the decision makers and everyone else just
listens for the results. Also, true fault tolerance can only be
achieved with a minimum of 1 disk write for every decision made.
That's usually the bottleneck for Paxos implementations but it can be
mitigated somewhat by doing multiple, parallel decisions and batching
the disk writes. If you're interested in playing around with it, the
zpax project (https://github.com/cocagne/zpax) does Paxos on top of
0MQ. I cleaned up the code not too long ago so it should be fairly
easy to follow.

Tom

On Tue, Feb 19, 2013 at 3:11 PM, Ian Barber <ian.barber at gmail.com> wrote:
>
>
> On Tue, Feb 19, 2013 at 9:07 PM, Pieter Hintjens <ph at imatix.com> wrote:
>>
>> Yes, events != state. In fact Zyre is already a distributed event bus
>> that can scale to about 100 peers (from experience) on WiFi, and a bit
>> more than that on a LAN. Persistence and consistency are quite another
>> story.
>>
>> -Pieter
>
>
> Tom's Paxos implementation might be of interest to the OP here as well:
> https://github.com/cocagne/paxos - layering something like that over the top
> of zyre might help. Or the post-ordering rotating token algo, treat zyre
> like a multicast.
>
> Ian



More information about the zeromq-dev mailing list