7 ms·

Myth: PageRank was the secret to Google's success It wasn't really that simple. For a brief while, perhaps it made a difference, but within perhaps a span of

by bborud 7y ago

Myth: PageRank was the secret to Google's success It wasn't really that simple. For a brief while, perhaps it made a difference, but within perhaps a span of 6 months, every decent search engine implemented page rank in one form or another. It is a cute story for the muggles to focus on. In reality search was already then about balancing a large number of signals into a decent ranking formula. It was much, much harder than just applying some magic algorithm and I think the people who built Google search back in those days deserve a lot more credit. But that wasn't really a sexy story, I guess. To a much greater degree than any algorithm or formula: Google's ability to execute, and to do so in cultural sympathy with the web, was more important. Much has been said about Google's Not Invented Here, but this made all the difference in the early days: you had to get things to where you could iterate and innovate fast. And I say that admiringly as someone who worked for one of their competitors at the time. I used to be jealous of Google because they were managed by people who were part of the Internet. Our management was alien to the Internet (and our business model was to power search for portals mainly run by horrible, stupid people in suits). Google was the only search engine that was properly in tune with its audience: focusing on the user. (Disclosure: I worked for FAST, then Yahoo, then Google until I quit Google in 2009)

baccheion 7y ago
Google's success was due to marketing and the opportunity to do so at massive scale (wide reach of and frequent media coverage). Even PageRank's mention was marketing, though the world at large had no idea what it meant. Most latched on to Google due to marketing surrounding and due to their IPO. Many heard and understood: money, billions, billionaire, slides and bicycles, etc. After, continued marketing kept it all going. Powerful illusion. Google is the embodiment of candy coated BS. Even now, they mainly continue due to continually marketing themselves as greatness.. ..and paying for Chrome, Android, and first placement on iPhones. They even had internal studies showing that while they are pervasive (most use at least one of their products), there isn't any stickiness. That is, if some other search engine had first placement on iPhones, the majority would be using that one. It's like the site that previously appeared when searching for a definition (dictionary.com?): while many used it and did so frequently, they often didn't even realize where they were. The world (and internet, even) at large is very different from the handful who think themselves aligned with the masses (ie, source of revenue). It's even funnier, as most thinking anyone cared about PageRank don't buy anything. No spending = you don't exist. Anything else is a coincidental nod, stupidity, or coincidence.
- remote_phone 7y ago
  Yeah, this is completely wrong. I watched Google grow up having worked in the area and they didn’t have any marketing at the beginning. They first became the best search engine many years before they started adding ads and well before their IPO.
  - baccheion 7y ago
    Being the best search engine was irrelevant to their rise. It's just a thing that happened to also (supposedly) be there. Supposedly, as its ranking algorithm was so heavily gamed by 2007 (already risen; popularity incentivized effort to game) it was a complete joke. Powerful illusion again, as they just covered it up and moved on. Also, internal tests from around 2009-2010 showed Bing was seen by users as producing better-quality results. Not only did they not have much marketing at the beginning, they also didn't have many users.
    - bborud 7y ago
      No, you are entirely wrong. There's no part of your "analysis" that has any relation to reality. I don't understand why you keep insisting. (I worked for two of Google's competitors in the years where they grew from a student project to a huge business. I had the opportunity to take a peek of the code of two other competitors. Many of my former colleagues helped build Bing. I also worked at Google for a few years. I was there so I would know something about it)
    - remote_phone 7y ago
      You literally have no idea what you’re talking about and spreading lies. Please stop this.
  - Twirrim 7y ago
    They did it by focusing on an absolutely minimal experience. The main search engines of the time, like Altavista, gave you a busy, cluttered, experience as they pursued monetisation. http://web.archive.org/web/20000308224033/http://www.altavista.com/ http://web.archive.org/web/20000308224033/http://www.altavis... Google gave you a search box: http://web.archive.org/web/20000815052943/http://www.google.com/ http://web.archive.org/web/20000815052943/http://www.google.... Over time they stripped even that down. Just as significantly, the search results were also uncluttered, and "good enough". This approach was absolutely critical when they started out, when people would most often be using 56k modems, where every byte had a real impact on the end user experience.
utopcell 7y ago
Not sure what you think you understood while at Yahoo, but Yahoo never used PageRank. Inlink-based signals, obviously, but never PageRank. Also, to state the obvious: Google's own execution of PageRank is orders of magnitude more complicated than what's been published. I fully agree with the greater point you are making however. Google is still at heart an engineering excellence company.
- throwawaymath 7y ago
  If you literally mean Yahoo never used the exact specification of the PageRank algorithm, that's probably correct. But if you mean PageRank conceptually, which includes any general search algorithm based on a discrete time Markov process, that's incorrect. In 2004 Yahoo acquired several smaller companies which were working on algorithmic search and page linking around the same time that Google was. They released their own ranking algorithm called WebRank which was substantially based on the methods of PageRank.
  - utopcell 7y ago
    Yahoo's core search engine was based on Inktomi's, which was acquired in 2003. There was no PageRank in there, and there was no infrastructure to execute anything similar either. (I worked for a significant amount of time on link-level features during Marissa's rebirth of Yahoo Search.) PageRank is an algorithm that is trivial to understand and prototype, but hard to scale efficiently to 100's of billions of pages.
    - bborud 7y ago
      That's largely true. But not quite, as Yahoo also acquired the FAST engine. However, since the team developing this were located in Trondheim, Norway, the FAST-based web search was eventually discontinued (and morphed into the VESPA project). FAST's web search used Page Rank from late 1999.
      - utopcell 7y ago
        Fascinating! This explains why I never saw anything like PageRank in YST: I presume it was present in what became Vespa (which, to be fair, probably didn't scale to YST's corpora sizes.) Pity we can't continue this conversation offline..
retreatguru 7y ago
What is the best way to learn how to execute at a very high level as Google does? We are a 25 person tech startup and while our culture is amazing we struggle with execution. This is an ongoing struggle for us; it’s so easy for a young startup to misunderstand the importance of execution. Is reading books enough, if so, which? Should we hire a COO from a company with a history of excellent execution (how to tell?). Are there courses to take? Or is it just about prioritizing excellent execution with continuous learning? Some resources that have helped so far: Scaling Up (book) and First Round Review blog.
- revvx 7y ago
  IME the thing that matters the most is focus, both on the micro and macro level. It's way too easy to get caught on things that won't make an iota of difference in your future. Focus is the first thing that goes out the window as soon a a company reach just a bit of momentum. I've seen several startups suffering from not being able to decide what they were. Engineering teams fractured because the company wanted half of them working on their bottom line and the other half working in some offshoot product. Even gigantic companies need that: notice how people criticize Google for creating and killing way too many products, and at the same time praising their minimalist webpage (since 1998), early GMail, etc. Same with Apple when Jobs returned to Apple and streamlined their product line, etc.
  - mikekchar 7y ago
    I agree with this but want to add prioritisation is key. As per Fred Brooks, there is no silver bullet. What he meant by that is that you can't get an order of magnitude more work done using a tool or process because in programming, 25% of your time is spent doing analysis and there is no way to reduce that (To me, that's only if you are executing well!). This generalises to a lot of other pursuits. Prioritisation is about deciding what not to do. Forget the BS story about putting rocks and sand in a jar, where the secret is to put the big rocks in first so that everything fits. That's not how you need to do prioritisation, because order does not appreciable change the amount of time something takes. The secret is to put only the big rocks in. Period. You go 6 times faster because you have 6 things you could do, but you only do one of them. Now the real kicker is that the only way to determine which one thing to do is to do analysis on all 6. An HN post is not really a reasonable way to describe this. However, consider a requirements discovery to be like a tree. Requirements are discovered at a particular rate, as you work on something. Discovery of one requirement leads to discovery of new requirements. It's a feedback loop. Pruning the tree as early as you can leads to significant gains later on. So while you can't actually get the 6x development time by avoiding 5/6ths of the requirements, you can pretty easily get 2-3x gains. BTW, for anyone interested in a more rigorous approach, consider taking something like Littlewood's model of defect discovery and assuming that requirements discovery has a similar curve. Littlewood's model is very naive, but I've found that it still has a lot of value. Again, sorry for cryptic hints here, but I don't have time to write a book on it (which it would certainly take...)

by bborud 7y ago

baccheion 7y ago

Google's success was due to marketing and the opportunity to do so at massive scale (wide reach of and frequent media coverage). Even PageRank's mention was marketing, though the world at large had no idea what it meant. Most latched on to Google due to marketing surrounding and due to their IPO. Many heard and understood: money, billions, billionaire, slides and bicycles, etc. After, continued marketing kept it all going. Powerful illusion. Google is the embodiment of candy coated BS. Even now, they mainly continue due to continually marketing themselves as greatness.. ..and paying for Chrome, Android, and first placement on iPhones. They even had internal studies showing that while they are pervasive (most use at least one of their products), there isn't any stickiness. That is, if some other search engine had first placement on iPhones, the majority would be using that one. It's like the site that previously appeared when searching for a definition (dictionary.com?): while many used it and did so frequently, they often didn't even realize where they were. The world (and internet, even) at large is very different from the handful who think themselves aligned with the masses (ie, source of revenue). It's even funnier, as most thinking anyone cared about PageRank don't buy anything. No spending = you don't exist. Anything else is a coincidental nod, stupidity, or coincidence.

remote_phone 7y ago
Yeah, this is completely wrong. I watched Google grow up having worked in the area and they didn’t have any marketing at the beginning. They first became the best search engine many years before they started adding ads and well before their IPO.
- baccheion 7y ago
  Being the best search engine was irrelevant to their rise. It's just a thing that happened to also (supposedly) be there. Supposedly, as its ranking algorithm was so heavily gamed by 2007 (already risen; popularity incentivized effort to game) it was a complete joke. Powerful illusion again, as they just covered it up and moved on. Also, internal tests from around 2009-2010 showed Bing was seen by users as producing better-quality results. Not only did they not have much marketing at the beginning, they also didn't have many users.
  - bborud 7y ago
    No, you are entirely wrong. There's no part of your "analysis" that has any relation to reality. I don't understand why you keep insisting. (I worked for two of Google's competitors in the years where they grew from a student project to a huge business. I had the opportunity to take a peek of the code of two other competitors. Many of my former colleagues helped build Bing. I also worked at Google for a few years. I was there so I would know something about it)
  - remote_phone 7y ago
    You literally have no idea what you’re talking about and spreading lies. Please stop this.
- Twirrim 7y ago
  They did it by focusing on an absolutely minimal experience. The main search engines of the time, like Altavista, gave you a busy, cluttered, experience as they pursued monetisation. http://web.archive.org/web/20000308224033/http://www.altavista.com/ http://web.archive.org/web/20000308224033/http://www.altavis... Google gave you a search box: http://web.archive.org/web/20000815052943/http://www.google.com/ http://web.archive.org/web/20000815052943/http://www.google.... Over time they stripped even that down. Just as significantly, the search results were also uncluttered, and "good enough". This approach was absolutely critical when they started out, when people would most often be using 56k modems, where every byte had a real impact on the end user experience.

utopcell 7y ago

Not sure what you think you understood while at Yahoo, but Yahoo never used PageRank. Inlink-based signals, obviously, but never PageRank. Also, to state the obvious: Google's own execution of PageRank is orders of magnitude more complicated than what's been published. I fully agree with the greater point you are making however. Google is still at heart an engineering excellence company.

throwawaymath 7y ago
If you literally mean Yahoo never used the exact specification of the PageRank algorithm, that's probably correct. But if you mean PageRank conceptually, which includes any general search algorithm based on a discrete time Markov process, that's incorrect. In 2004 Yahoo acquired several smaller companies which were working on algorithmic search and page linking around the same time that Google was. They released their own ranking algorithm called WebRank which was substantially based on the methods of PageRank.
- utopcell 7y ago
  Yahoo's core search engine was based on Inktomi's, which was acquired in 2003. There was no PageRank in there, and there was no infrastructure to execute anything similar either. (I worked for a significant amount of time on link-level features during Marissa's rebirth of Yahoo Search.) PageRank is an algorithm that is trivial to understand and prototype, but hard to scale efficiently to 100's of billions of pages.
  - bborud 7y ago
    That's largely true. But not quite, as Yahoo also acquired the FAST engine. However, since the team developing this were located in Trondheim, Norway, the FAST-based web search was eventually discontinued (and morphed into the VESPA project). FAST's web search used Page Rank from late 1999.
    - utopcell 7y ago
      Fascinating! This explains why I never saw anything like PageRank in YST: I presume it was present in what became Vespa (which, to be fair, probably didn't scale to YST's corpora sizes.) Pity we can't continue this conversation offline..

retreatguru 7y ago

What is the best way to learn how to execute at a very high level as Google does? We are a 25 person tech startup and while our culture is amazing we struggle with execution. This is an ongoing struggle for us; it’s so easy for a young startup to misunderstand the importance of execution. Is reading books enough, if so, which? Should we hire a COO from a company with a history of excellent execution (how to tell?). Are there courses to take? Or is it just about prioritizing excellent execution with continuous learning? Some resources that have helped so far: Scaling Up (book) and First Round Review blog.

revvx 7y ago
IME the thing that matters the most is focus, both on the micro and macro level. It's way too easy to get caught on things that won't make an iota of difference in your future. Focus is the first thing that goes out the window as soon a a company reach just a bit of momentum. I've seen several startups suffering from not being able to decide what they were. Engineering teams fractured because the company wanted half of them working on their bottom line and the other half working in some offshoot product. Even gigantic companies need that: notice how people criticize Google for creating and killing way too many products, and at the same time praising their minimalist webpage (since 1998), early GMail, etc. Same with Apple when Jobs returned to Apple and streamlined their product line, etc.
- mikekchar 7y ago
  I agree with this but want to add prioritisation is key. As per Fred Brooks, there is no silver bullet. What he meant by that is that you can't get an order of magnitude more work done using a tool or process because in programming, 25% of your time is spent doing analysis and there is no way to reduce that (To me, that's only if you are executing well!). This generalises to a lot of other pursuits. Prioritisation is about deciding what not to do. Forget the BS story about putting rocks and sand in a jar, where the secret is to put the big rocks in first so that everything fits. That's not how you need to do prioritisation, because order does not appreciable change the amount of time something takes. The secret is to put only the big rocks in. Period. You go 6 times faster because you have 6 things you could do, but you only do one of them. Now the real kicker is that the only way to determine which one thing to do is to do analysis on all 6. An HN post is not really a reasonable way to describe this. However, consider a requirements discovery to be like a tree. Requirements are discovered at a particular rate, as you work on something. Discovery of one requirement leads to discovery of new requirements. It's a feedback loop. Pruning the tree as early as you can leads to significant gains later on. So while you can't actually get the 6x development time by avoiding 5/6ths of the requirements, you can pretty easily get 2-3x gains. BTW, for anyone interested in a more rigorous approach, consider taking something like Littlewood's model of defect discovery and assuming that requirements discovery has a similar curve. Littlewood's model is very naive, but I've found that it still has a lot of value. Again, sorry for cryptic hints here, but I don't have time to write a book on it (which it would certainly take...)