4 ms·

The Google technology stack (2008)

68 points by erbdex 13y ago

igvadaimon 13y ago
http://webcache.googleusercontent.com/search?q=cache:http://michaelnielsen.org/blog/lecture-course-the-google-technology-stack/ http://webcache.googleusercontent.com/search?q=cache:http://...
sherbet 13y ago
Circa ~2005; correct me if I'm wrong but I don't think MapReduce has been used for search indexing at Google for half a decade.
- erbdex 13y ago
  It is more about the approach that keeps them there. i quote- "Part of what makes Google such an amazing engine of innovation is their internal technology stack: a set of powerful proprietary technologies that makes it easy for Google developers to generate and process enormous quantities of data. According to a senior Microsoft developer who moved to Google, Googlers work and think at a higher level of abstraction than do developers at many other companies, including Microsoft: “Google uses Bayesian filtering the way Microsoft uses the if statement”
- natch 13y ago
  What replaced it?
  - nostrademons 13y ago
    Google Caffeine was the public release. I don't believe the technical details have been published yet.
  - teraflop 13y ago
    Caffeine, based on the Percolator incremental processing framework: http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/36726.pdf http://static.googleusercontent.com/media/research.google.co...
    - natch 13y ago
      Thanks!
xrocka 13y ago
next time you should host youre site on appengine... :)
- erbdex 13y ago
  +1 The HN effect.
- ibudiallo 13y ago
  Maybe wordpress is to blame, but the poor man can still handle HN traffic. Did it recently [1]. Disclaimer: it's my blog. [1]: http://idiallo.com/blog/2013/11/handling-1-million-web-request http://idiallo.com/blog/2013/11/handling-1-million-web-reque...
tedchs 13y ago
This is about a lecture that was going to be held in 2008. Can we update the HN subject line to reflect this?
- nostrademons 13y ago
  Also Google's publications tend to run 3-4 years behind when the technology is actually developed. So this is actually describing the Google stack as it existed c. 2004.
  - judk 13y ago
    3-4 years sounds like how long it takes for a new technology to be deployed as a major part of the stack
    - nostrademons 13y ago
      3-4 years after deployment. It takes time to understand how a technology will be used and for the details of it to stabilize.

68 points by erbdex 13y ago

http://webcache.googleusercontent.com/search?q=cache:http://michaelnielsen.org/blog/lecture-course-the-google-technology-stack/ http://webcache.googleusercontent.com/search?q=cache:http://...

sherbet 13y ago

Circa ~2005; correct me if I'm wrong but I don't think MapReduce has been used for search indexing at Google for half a decade.

erbdex 13y ago
It is more about the approach that keeps them there. i quote- "Part of what makes Google such an amazing engine of innovation is their internal technology stack: a set of powerful proprietary technologies that makes it easy for Google developers to generate and process enormous quantities of data. According to a senior Microsoft developer who moved to Google, Googlers work and think at a higher level of abstraction than do developers at many other companies, including Microsoft: “Google uses Bayesian filtering the way Microsoft uses the if statement”
natch 13y ago
What replaced it?
- nostrademons 13y ago
  Google Caffeine was the public release. I don't believe the technical details have been published yet.
- teraflop 13y ago
  Caffeine, based on the Percolator incremental processing framework: http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/36726.pdf http://static.googleusercontent.com/media/research.google.co...
  - natch 13y ago
    Thanks!

xrocka 13y ago

next time you should host youre site on appengine... :)

erbdex 13y ago
+1 The HN effect.
ibudiallo 13y ago
Maybe wordpress is to blame, but the poor man can still handle HN traffic. Did it recently [1]. Disclaimer: it's my blog. [1]: http://idiallo.com/blog/2013/11/handling-1-million-web-request http://idiallo.com/blog/2013/11/handling-1-million-web-reque...

tedchs 13y ago

This is about a lecture that was going to be held in 2008. Can we update the HN subject line to reflect this?

nostrademons 13y ago
Also Google's publications tend to run 3-4 years behind when the technology is actually developed. So this is actually describing the Google stack as it existed c. 2004.
- judk 13y ago
  3-4 years sounds like how long it takes for a new technology to be deployed as a major part of the stack
  - nostrademons 13y ago
    3-4 years after deployment. It takes time to understand how a technology will be used and for the details of it to stabilize.