5 ms·

> Google's origin story doesn't seem relevant to me, in that Google won because they made radical leaps forward in search quality, compute cost, and and compute

> Google's origin story doesn't seem relevant to me, in that Google won because they made radical leaps forward in search quality, compute cost, and and compute scale. That's the myth, but it's pretty far from the reality. Long before Google became such a dominant player, competing search engines had matched or exceeded its quality, compute cost, and ability to scale. > All at a time when the web was exploding: only 19% of people used the web at Google's launch; by 2008 it was circa 70%. Bingo. You nailed it. It turns out the search market was largely a distribution game. You wanted to be the search engine distributed to that massively growing audience, and Google did a great job of establishing itself as the first search engine presented to people. > It seems to me that by 2008, there were fewer easy innovations lying around to make. There's certainly been a lot of innovations in search since 2008, but nothing that was "disruptive". It's very hard to get more than a small percentage of people to change what search engine they use, and advertisers are not particularly interested in chasing that segment of the population. > The way that other people have tried and failed since points that way too, as Google's market share is even larger now than it was in 2008. Yeah, I really have to scratch my head at how people look at the empirical evidence and think that in 2008 it was a mistake to recognize the losing position competitors were in.

Bukhmanizer 6y ago
> Long before Google became such a dominant player, competing search engines had matched or exceeded its quality, compute cost, and ability to scale. Just curious, which ones? I remember Google felt so far above any other search engine for a very long time.
- hogFeast 6y ago
  Essentially all of their tech was taken from Robin Li and AltaVista/DEC. Brin did a lot of the implementation work, Page did nothing (a lot of the work he did do, didn't scale, and was highly inferior to existing work as he wasn't a strong programmer...I think he provided a server room or something, he was the snake oil salesman). The actual technological innovation in the search product was minimal (the stuff they did later to scale was pure innovation, but that came much later). They largely purchased their position (for a good price).
  - deleted 6y ago
    [deleted]
  - cbsmith 6y ago
    The tech was "stolen" from HITS/IBM Research. Because of their success with their initial data center approach AltaVista was arguably one of the last to shift to a model that more closely resembled the one Google employed, but they certainly did a very fine job of it (late mover advantage and all that). I still liked AllTheWeb's data centers somewhat better, but at that point the differences were pretty small. I wouldn't agree with your characterization of Brin & Page's contributions, but their direct technical contributions to Google by that time were at best marginally greater than Yang's, so it's not really relevant anyway. Google definitely had some great tech, but the extent to which that won them the game is more myth than reality. Their aggressive strategic moves are really what made them successful.
    - cromwellian 6y ago
      I worked at IBM TJ Watson research at the time. It's hard to "steal" from IBM, because at the time I was there, IBM's #1 problem was getting its research into the world. In 1996, I worked in a group on digital cash payment schemes, digital checking, years before PayPal, they had a trusted enclave (CITADEL PCMCIA card), we had a MIME plugin for Mosaic and Unix email that would do peer to peer digital money transfers, and when it came time to ask "when are we going to try to productize this", the answer came back "we talked with banks and will be ready to do a pilot project in 5 years" This is much like Steve Jobs "stealing" GUI, Smalltalk, and Ethernet from PARC and claiming there was no innovation. At that stage, shipping something consumers can use is innovation. If you were a techie/engineer in 1998, as opposed to a general consumer (of which there were very few), Google results WERE better than AltaVista and competitors. The early adopters and pioneers switched because like with Chrome v1, a lot of the value proposition was obvious. The bigger question perhaps is why didn't their competitors react quickly to prevent their rise, and I think the answer to that is in general, once a company makes a large splash, and gets the public attention, it's hard for others to overcome the free value from being the thought leader. This is true of Apple and true of Tesla in cars. Tesla makes the best EV right now. It's sexy, it's fast, and Elon Musk commands the world's attention. One day, other brands will catch up, they'll be Tesla competitors just as good, but no one will care, and Tesla -- if Musk doesn't fsck it up -- will continue to have a halo effect, a reality distortion field.
- cbsmith 6y ago
  Inktomi, Altavista, AllTheWeb, all later under Yahoo. No question about the "feel", but it's amazing how perception fuels reality. We did a lot of studies of this, and one of the more interesting ones was that if you told people they were seeing Google search results (regardless if they were), they'd be more inclined to rewrite their queries and look farther down in the search results, rather than try using another search engine. Beyond the obvious revenue & marketshare benefits, this helped Google two-fold technically: it gave them better data on query rewrites and search results. The hoops you had to jump through to match our exceed their performance were substantial.
  - JohnBooty 6y ago
    We did a lot of studies of this, and one of the more interesting ones was that if you told people they were seeing Google search results This is super-interesting; where might one read more about these studies? Also, when were the studies done? I'm curious about the "when" because I think Google's reputation has slipped quite a bit.
    - cbsmith 6y ago
      These were done around 2004-2006 IIRC, I don't know if the brand impact today is the same as it was back then. I'll see if I can find publications, but I think most, if not all of it, was not published. Even when you told people that we were substituting in different search engines and the presentation and layout was not indicative of where the results came from, putting a Google logo on the page produced a measurable difference in behaviour -far more so than changes to the results themselves (admittedly, we never presented ridiculous results, like say results from an entirely different query or a just random landing pages, but we did present them with, for example, purely paid search results). In general though, consumer confirmation bias is not that surprising, right? Particularly with technology brands, it's pretty well established that confidence in the brand leads one to be more likely to attribute failure to one's own "mistakes" and therefore invest in further exploration of the product and/or modifying one's interactions with it. It also diminishes one's belief that another product would produce better results, therefore diminishing interest in exploring alternatives; it also leads one to perceive a product's performance as "better" when it is exactly the same.
coryrc 6y ago
I was an AltaVista user. Then I started having to use a meta search. Then Google came along and it always worked. Who cares if the others caught up? I had no reason to look for over a decade. But now I feel the amount of money to be made means search will never be good again: you can pay people pennies to get a few more pennies for yourself, just to bury the better stuff given freely.
- cbsmith 6y ago
  I think you accurately captured the reality of the context. For the most part, as long as people perceive they are getting a good search result, they aren't to look at other options, let alone switch.
chaostheory 6y ago
> That's the myth, but it's pretty far from the reality. Long before Google became such a dominant player, competing search engines had matched or exceeded its quality, compute cost, and ability to scale. I disagree. Inktomi, Altavista, and AllTheWeb weren't even close in terms of search result revelance and the number of pages they indexed. Every time they attempted to close the gap, Google would just widen it with stuff like real time search, their revolutionary maps UI, and gmail.
- cbsmith 6y ago
  You're certainly welcome to disagree, but Google would be the first to tell you how relevant the number of pages indexed would be. When you actually measured the result relevance number, it turned out that all the major engines were pretty similar, and Google often was not the top one. More importantly though, the observed phenomenon was that no matter how bad the results you presented were, most people didn't switch to another search engine. I get that you're a Google fan, but GMail & Maps weren't part of the search engine. Ironically "real time search" was rolled out as a PR move to counteract the narrative that was being constructed at the time: that Google was falling behind UX innovations in the search engine space. All of the above were rolled out after Google was the dominant player in the search space. Google has absolutely done some amazing things, and their search team is second to none, but the narrative that they became the dominant player because of their technical advantages is very misleading.
  - chaostheory 6y ago
    I am no longer a Google fan. Haven't been for years so there's no bias. > More importantly though, the observed phenomenon was that no matter how bad the results you presented were, most people didn't switch to another search engine. That may be the case today because the results are similar now, but it wasn't the same case years ago. The ancient search engines prior to Google weren't much better than human curated directories like Yahoo. Otherwise, why would anyone switch from them to Google?
    - cbsmith 6y ago
wpietri 6y ago
Thanks for the detailed reply. > Long before Google became such a dominant player, competing search engines had matched or exceeded its quality, compute cost, and ability to scale. Could you say more about this? I wasn't involved, but my recollection is that Google pioneered using a zillion low-cost nodes for search. E.g., I remember that AltaVista was using a big iron approach partly as a way of showing off DEC's wares. I can believe everybody had to try to keep up, but I am skeptical that Google slowed down enough for everybody else to catch up and/or surpass them.
- cbsmith 6y ago
  > Could you say more about this? I wasn't involved, but my recollection is that Google pioneered using a zillion low-cost nodes for search. E.g., I remember that AltaVista was using a big iron approach partly as a way of showing off DEC's wares. I can believe everybody had to try to keep up, but I am skeptical that Google slowed down enough for everybody else to catch up and/or surpass them. It's the nature of innovation that something that puts you ahead doesn't create a sustainable advantage (e.g. AltaVista as you highlighted), and there tends to be some marginal advantage for second movers of having the benefit of hindsight. I don't know that anyone knows the why of how this played out in Google's case, basically no one is privy to all the information about each search engine's operations, and to the extent they do, they likely cannot share what they know. Certainly, on the cost side, Google's approach wasn't entirely novel in the sense that there was a lot of expertise already out there on how to run low-cost nodes at scale. Reverse indexes are actually a pretty "embarrassingly parallel" problem, and Page Rank style algorithms don't really make it much more difficult. In short: once Google demonstrated the value of the approach, it was pretty easy to figure out how to implement it. Google certainly continued to innovate on their data center designs, but if anything were at a disadvantage in terms of rolling it out because they already had a substantial investment in their existing designs that were working pretty well for them. There were also startups like FAST (AllTheWeb) that sprung up after Google and were able to leapfrog Google much as Google leapfrogged its competitors. In terms of relevancy, we did do double-blind style evaluations of the relevancy of search engines as well as of the consumer behaviour with search engines. What we found was that by 2004-2006, search engine competitors had largely closed the relevancy gap, and in some cases reliably exceeded Google. What we also found though was that it largely didn't matter. Even when presented with terrible results, most of the population would not switch search engines. Indeed, most of the growth in market share amongst search engines came through distribution deals where users would be presented with a search engine as the default. Now, that's not to say that people didn't switch because they found a search engine to be better. They absolutely did.. but only 10-30% of the users would ever engage in switching behaviour for any reason. Getting those users certainly would help you, but the vast majority of the market was driven by other factors.

Bukhmanizer 6y ago

> Long before Google became such a dominant player, competing search engines had matched or exceeded its quality, compute cost, and ability to scale. Just curious, which ones? I remember Google felt so far above any other search engine for a very long time.

hogFeast 6y ago
Essentially all of their tech was taken from Robin Li and AltaVista/DEC. Brin did a lot of the implementation work, Page did nothing (a lot of the work he did do, didn't scale, and was highly inferior to existing work as he wasn't a strong programmer...I think he provided a server room or something, he was the snake oil salesman). The actual technological innovation in the search product was minimal (the stuff they did later to scale was pure innovation, but that came much later). They largely purchased their position (for a good price).
- deleted 6y ago
  [deleted]
- cbsmith 6y ago
  The tech was "stolen" from HITS/IBM Research. Because of their success with their initial data center approach AltaVista was arguably one of the last to shift to a model that more closely resembled the one Google employed, but they certainly did a very fine job of it (late mover advantage and all that). I still liked AllTheWeb's data centers somewhat better, but at that point the differences were pretty small. I wouldn't agree with your characterization of Brin & Page's contributions, but their direct technical contributions to Google by that time were at best marginally greater than Yang's, so it's not really relevant anyway. Google definitely had some great tech, but the extent to which that won them the game is more myth than reality. Their aggressive strategic moves are really what made them successful.
  - cromwellian 6y ago
    I worked at IBM TJ Watson research at the time. It's hard to "steal" from IBM, because at the time I was there, IBM's #1 problem was getting its research into the world. In 1996, I worked in a group on digital cash payment schemes, digital checking, years before PayPal, they had a trusted enclave (CITADEL PCMCIA card), we had a MIME plugin for Mosaic and Unix email that would do peer to peer digital money transfers, and when it came time to ask "when are we going to try to productize this", the answer came back "we talked with banks and will be ready to do a pilot project in 5 years" This is much like Steve Jobs "stealing" GUI, Smalltalk, and Ethernet from PARC and claiming there was no innovation. At that stage, shipping something consumers can use is innovation. If you were a techie/engineer in 1998, as opposed to a general consumer (of which there were very few), Google results WERE better than AltaVista and competitors. The early adopters and pioneers switched because like with Chrome v1, a lot of the value proposition was obvious. The bigger question perhaps is why didn't their competitors react quickly to prevent their rise, and I think the answer to that is in general, once a company makes a large splash, and gets the public attention, it's hard for others to overcome the free value from being the thought leader. This is true of Apple and true of Tesla in cars. Tesla makes the best EV right now. It's sexy, it's fast, and Elon Musk commands the world's attention. One day, other brands will catch up, they'll be Tesla competitors just as good, but no one will care, and Tesla -- if Musk doesn't fsck it up -- will continue to have a halo effect, a reality distortion field.
cbsmith 6y ago
Inktomi, Altavista, AllTheWeb, all later under Yahoo. No question about the "feel", but it's amazing how perception fuels reality. We did a lot of studies of this, and one of the more interesting ones was that if you told people they were seeing Google search results (regardless if they were), they'd be more inclined to rewrite their queries and look farther down in the search results, rather than try using another search engine. Beyond the obvious revenue & marketshare benefits, this helped Google two-fold technically: it gave them better data on query rewrites and search results. The hoops you had to jump through to match our exceed their performance were substantial.
- JohnBooty 6y ago
  We did a lot of studies of this, and one of the more interesting ones was that if you told people they were seeing Google search results This is super-interesting; where might one read more about these studies? Also, when were the studies done? I'm curious about the "when" because I think Google's reputation has slipped quite a bit.
  - cbsmith 6y ago
    These were done around 2004-2006 IIRC, I don't know if the brand impact today is the same as it was back then. I'll see if I can find publications, but I think most, if not all of it, was not published. Even when you told people that we were substituting in different search engines and the presentation and layout was not indicative of where the results came from, putting a Google logo on the page produced a measurable difference in behaviour -far more so than changes to the results themselves (admittedly, we never presented ridiculous results, like say results from an entirely different query or a just random landing pages, but we did present them with, for example, purely paid search results). In general though, consumer confirmation bias is not that surprising, right? Particularly with technology brands, it's pretty well established that confidence in the brand leads one to be more likely to attribute failure to one's own "mistakes" and therefore invest in further exploration of the product and/or modifying one's interactions with it. It also diminishes one's belief that another product would produce better results, therefore diminishing interest in exploring alternatives; it also leads one to perceive a product's performance as "better" when it is exactly the same.

coryrc 6y ago

I was an AltaVista user. Then I started having to use a meta search. Then Google came along and it always worked. Who cares if the others caught up? I had no reason to look for over a decade. But now I feel the amount of money to be made means search will never be good again: you can pay people pennies to get a few more pennies for yourself, just to bury the better stuff given freely.

cbsmith 6y ago
I think you accurately captured the reality of the context. For the most part, as long as people perceive they are getting a good search result, they aren't to look at other options, let alone switch.

chaostheory 6y ago

> That's the myth, but it's pretty far from the reality. Long before Google became such a dominant player, competing search engines had matched or exceeded its quality, compute cost, and ability to scale. I disagree. Inktomi, Altavista, and AllTheWeb weren't even close in terms of search result revelance and the number of pages they indexed. Every time they attempted to close the gap, Google would just widen it with stuff like real time search, their revolutionary maps UI, and gmail.

cbsmith 6y ago
You're certainly welcome to disagree, but Google would be the first to tell you how relevant the number of pages indexed would be. When you actually measured the result relevance number, it turned out that all the major engines were pretty similar, and Google often was not the top one. More importantly though, the observed phenomenon was that no matter how bad the results you presented were, most people didn't switch to another search engine. I get that you're a Google fan, but GMail & Maps weren't part of the search engine. Ironically "real time search" was rolled out as a PR move to counteract the narrative that was being constructed at the time: that Google was falling behind UX innovations in the search engine space. All of the above were rolled out after Google was the dominant player in the search space. Google has absolutely done some amazing things, and their search team is second to none, but the narrative that they became the dominant player because of their technical advantages is very misleading.
- chaostheory 6y ago
  I am no longer a Google fan. Haven't been for years so there's no bias. > More importantly though, the observed phenomenon was that no matter how bad the results you presented were, most people didn't switch to another search engine. That may be the case today because the results are similar now, but it wasn't the same case years ago. The ancient search engines prior to Google weren't much better than human curated directories like Yahoo. Otherwise, why would anyone switch from them to Google?
  - cbsmith 6y ago

wpietri 6y ago

Thanks for the detailed reply. > Long before Google became such a dominant player, competing search engines had matched or exceeded its quality, compute cost, and ability to scale. Could you say more about this? I wasn't involved, but my recollection is that Google pioneered using a zillion low-cost nodes for search. E.g., I remember that AltaVista was using a big iron approach partly as a way of showing off DEC's wares. I can believe everybody had to try to keep up, but I am skeptical that Google slowed down enough for everybody else to catch up and/or surpass them.

cbsmith 6y ago
> Could you say more about this? I wasn't involved, but my recollection is that Google pioneered using a zillion low-cost nodes for search. E.g., I remember that AltaVista was using a big iron approach partly as a way of showing off DEC's wares. I can believe everybody had to try to keep up, but I am skeptical that Google slowed down enough for everybody else to catch up and/or surpass them. It's the nature of innovation that something that puts you ahead doesn't create a sustainable advantage (e.g. AltaVista as you highlighted), and there tends to be some marginal advantage for second movers of having the benefit of hindsight. I don't know that anyone knows the why of how this played out in Google's case, basically no one is privy to all the information about each search engine's operations, and to the extent they do, they likely cannot share what they know. Certainly, on the cost side, Google's approach wasn't entirely novel in the sense that there was a lot of expertise already out there on how to run low-cost nodes at scale. Reverse indexes are actually a pretty "embarrassingly parallel" problem, and Page Rank style algorithms don't really make it much more difficult. In short: once Google demonstrated the value of the approach, it was pretty easy to figure out how to implement it. Google certainly continued to innovate on their data center designs, but if anything were at a disadvantage in terms of rolling it out because they already had a substantial investment in their existing designs that were working pretty well for them. There were also startups like FAST (AllTheWeb) that sprung up after Google and were able to leapfrog Google much as Google leapfrogged its competitors. In terms of relevancy, we did do double-blind style evaluations of the relevancy of search engines as well as of the consumer behaviour with search engines. What we found was that by 2004-2006, search engine competitors had largely closed the relevancy gap, and in some cases reliably exceeded Google. What we also found though was that it largely didn't matter. Even when presented with terrible results, most of the population would not switch search engines. Indeed, most of the growth in market share amongst search engines came through distribution deals where users would be presented with a search engine as the default. Now, that's not to say that people didn't switch because they found a search engine to be better. They absolutely did.. but only 10-30% of the users would ever engage in switching behaviour for any reason. Getting those users certainly would help you, but the vast majority of the market was driven by other factors.