How Google Works"How Google Works" by David Carr, Basline Magazine, July 7, 2006
Google's compute system is actually a really big LISP machine: http://en.wikipedia.org/wiki/MapReduce
Google MapReduce, GFS, & more from the source: Google Labs papers
DIY GoogleSo if you want to make your own (initially small and cheap) version of Google (a Giggle?), how do you do it?
To get the compute power, Amazon EC2 is clearly the way to go.
To take advantage of massively parallel clusters, we need to write our applications in such a way that they can be processed in a parallel fashion. A number of traditional approaches are things like Globus Toolkit, MPI, and JavaSpaces. But if we want to do it the Google-way, then Hadoop is the most interesting because it is an implementation on Map-Reduce in Java as OSS (Apache).