[zeromq-dev] Simple GFS-like System
wolf at cs.cmu.edu
Mon May 16 05:21:02 CEST 2011
Thanks for the interest everyone :-)
This really grew out of some frustration with performance issues (and
configuration complexity!) I was seeing using KFS and HDFS on a small
Eucalyptus cluster (inside VMs). I've also seen performance issues on bare
I decided I wanted something dead simple that I could reason about and get
baseline numbers out of for reading and writing into a cluster.
Turns out ZeroMQ + my simple implementation on top of Redis can out perform
them in read/write performance (more evaluation I hope to put on the web in
the near future).
On Sat, May 14, 2011 at 11:21 AM, Thomas S Hatch <thatch45 at gmail.com> wrote:
> On Sat, May 14, 2011 at 12:41 AM, Pieter Hintjens <ph at imatix.com> wrote:
>> On Sat, May 14, 2011 at 8:37 AM, Wolfgang Richter <wolf at cs.cmu.edu>
>> > Backed by ZeroMQ (right now, only REQ-REP sockets; planning on using
>> > PUSH-PULL for pipelining later) and Redis (for metadata storage):
>> > https://github.com/theonewolf/TripleD/
>> Neat! I've sent a tweet to my three loyal followers.
>> It's really great to see ZeroMQ being used as the transport layer for
>> smart new projects like this. If you want maximum visibility for
>> Triple-D, I'd suggest writing a blog posting that explains your goals,
>> and also acts as a simple tutorial for ZeroMQ.
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
> This is a great concept! I use MooseFS a great deal in my datacenters and
> over and over again I say "we need this written with ZeroMQ and to use redis
> as the metadata back end"! You are on the right track, and I hope to see
> this grow into something more powerful!
> -Thomas S Hatch
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the zeromq-dev