5 ms·

I enjoyed that, but I'm not sure about a couple of the claims. I switched from AltaVista to Google because it gave me better results almost all of the time - i

I enjoyed that, but I'm not sure about a couple of the claims. I switched from AltaVista to Google because it gave me better results almost all of the time - if that weren't the case I, and however many millions of others, wouldn't have switched. The 'expensive data centre theory' may have sped-up the demise of AV et al, but I don't think it's fair to say Google succeeded by having low costs rather than a superior product. I'd also like to see data to back up the claim that "The vast majority of users are no longer clicking through pages of Google results". Again, not my experience, but I recognise that I am a datapoint of one. I do note that increasingly Google's own answers (especially maps and images) are providing me with the direction I need in response to a search query, but even then I usually click through. Edit: Re-read that. At first I thought it meant clicking through to the pages that are returned as results; it may mean not clicking through the pages of Google results (1-infinity below). Still, I thought <5% of people ever clicked through to the second page (most people refined their search if the first results weren't what they wanted) so I'm not sure if anything has changed. I think Google search is damaged, not yet a product failure but not yet "no longer" a problem.

saurik 16y ago
How much of Google returning better results than AltaVista, though, came down to Google being able to expand its index with new pages faster? What I remember is exactly what the article claims: that competing search engines couldn't "keep up" with Google's increasing scale.
- petervandijck 16y ago
  I remember Google mostly for having good results, yes, but more for being FAST. It was just soo fast, when all other search engines took up to seconds to show results.
- michael_dorfman 16y ago
  My experience, or at least my recollection, was quite different. I had no complaints whatsoever with AltaVista not adding pages to the index quickly enough; rather, it is that the results increasingly appeared to be random pages that happened to contain the search term, and you might need to wade through several pages of results to find what you were looking for. With Google, the page you wanted seemed to almost uncannily be located toward the top of the list. At the time, it seemed like magic.
  - deleted 16y ago
    [deleted]
  - aneth 16y ago
    I agree. The claim that other search engines couldn't keep up does not gel with my recollection, and is probably being made by people who were not searching the web daily in 1998. Google won because they were good at relevance, not index size. In fact, I recall AltaVista was the go to site for obscure pages that google had not indexed.
- randall 16y ago
  I distinctly remember 404s on Yahoo, which was why I switched.
diego 16y ago
I was at Inktomi in the late 90s/early 2000s. We were trying to keep up with Google in terms of scale, and we could do everything they could do in terms of link analysis, relevance, etc. Objectively, our results were as good as theirs. They took over in terms of index size because they could add cheap Linux machines easily. We were tied to our expensive Sun hardware, it took us three years to switch to Linux (long story) and then it was too late. tl,dr: relevance doesn't matter if you don't have the result the user wants.
- RyanMcGreal 16y ago
  Google's build-out has been nothing short of astonishing. I remember when Google indexed the web once a month - and the web was much smaller then. Now my sites get crawled several times a day - and new articles on my news site are in Google News within less than an hour. The mind boggles.
  - diego 16y ago
    The main reason their cache was useful back then is that many pages were long gone and the links were 404s. Inktomi marketing always tried to play up our freshness compared to Google. In 1999 they had some pages in their cache that were 3-5 months old. We were pushing new indexes once a week, no page was older than a month. Seems quaint now.
    - nostrademons 16y ago
      I find that the cache is useful now because articles get posted to social news sites, the influx of traffic brings down the site, and Google Cache is a handy mirror that usually seems to have the page already.
  - candeira 16y ago
    I just did a search for an error message in a piece of software. The one result from Google was my own pastebinned traceback from five hours earlier. Google indexes random pastebins faster than I can forget I posted them. This still cracks me up, but now I laugh nervously: http://www.ftrain.com/robot_exclusion_protocol.html http://www.ftrain.com/robot_exclusion_protocol.html
jobu 16y ago
Swu said it best: "Google lacks a feature that it should have added year ago: A search user who is logged in should have the ability to block entire domains from all future results. The benefits of this are many. The cost is very low. Why is this option not already available? Google - we depend on you. Do it."
- mistermann 16y ago
  >Why is this option not already available? I think the answer to this question is the same reason why google's results have become spammy. Allowing users to exclude specific domains, or even having a "report as content farm" button, are in direct conflict with google's business model, to a certain degree.
  - kijinbear 16y ago
    Maybe Google will charge money for the "premium" search service. Take money from advertisers to show their ads, _and_ take money from users to hide the same ads. I'll hate Google when that day comes. At least I take comfort in the fact that programming-related searches now return more StackOverflow links than ExpertSexChange links.
  - gwern 16y ago
    I wonder how much is due to user-friendliness. An invisible global blacklist on my search results? How could that possibly go wrong...
    - quanticle 16y ago
      Well, does it have to be an invisible global blacklist? Would it be possible to create personal blacklists and have one for each user? I mean, Google already customizes search results for logged in users, so this wouldn't be too far of a stretch.
      - gwern 16y ago
        > Would it be possible to create personal blacklists and have one for each user? I think that's pretty clearly what we were talking about. There are invisible global blacklists already, as should come as no surprise (even if you haven't run into one of the hits omitted thanks to the DMCA).

saurik 16y ago

How much of Google returning better results than AltaVista, though, came down to Google being able to expand its index with new pages faster? What I remember is exactly what the article claims: that competing search engines couldn't "keep up" with Google's increasing scale.

petervandijck 16y ago
I remember Google mostly for having good results, yes, but more for being FAST. It was just soo fast, when all other search engines took up to seconds to show results.
michael_dorfman 16y ago
My experience, or at least my recollection, was quite different. I had no complaints whatsoever with AltaVista not adding pages to the index quickly enough; rather, it is that the results increasingly appeared to be random pages that happened to contain the search term, and you might need to wade through several pages of results to find what you were looking for. With Google, the page you wanted seemed to almost uncannily be located toward the top of the list. At the time, it seemed like magic.
- deleted 16y ago
  [deleted]
- aneth 16y ago
  I agree. The claim that other search engines couldn't keep up does not gel with my recollection, and is probably being made by people who were not searching the web daily in 1998. Google won because they were good at relevance, not index size. In fact, I recall AltaVista was the go to site for obscure pages that google had not indexed.
randall 16y ago
I distinctly remember 404s on Yahoo, which was why I switched.

diego 16y ago

I was at Inktomi in the late 90s/early 2000s. We were trying to keep up with Google in terms of scale, and we could do everything they could do in terms of link analysis, relevance, etc. Objectively, our results were as good as theirs. They took over in terms of index size because they could add cheap Linux machines easily. We were tied to our expensive Sun hardware, it took us three years to switch to Linux (long story) and then it was too late. tl,dr: relevance doesn't matter if you don't have the result the user wants.

RyanMcGreal 16y ago
Google's build-out has been nothing short of astonishing. I remember when Google indexed the web once a month - and the web was much smaller then. Now my sites get crawled several times a day - and new articles on my news site are in Google News within less than an hour. The mind boggles.
- diego 16y ago
  The main reason their cache was useful back then is that many pages were long gone and the links were 404s. Inktomi marketing always tried to play up our freshness compared to Google. In 1999 they had some pages in their cache that were 3-5 months old. We were pushing new indexes once a week, no page was older than a month. Seems quaint now.
  - nostrademons 16y ago
    I find that the cache is useful now because articles get posted to social news sites, the influx of traffic brings down the site, and Google Cache is a handy mirror that usually seems to have the page already.
- candeira 16y ago
  I just did a search for an error message in a piece of software. The one result from Google was my own pastebinned traceback from five hours earlier. Google indexes random pastebins faster than I can forget I posted them. This still cracks me up, but now I laugh nervously: http://www.ftrain.com/robot_exclusion_protocol.html http://www.ftrain.com/robot_exclusion_protocol.html

jobu 16y ago

Swu said it best: "Google lacks a feature that it should have added year ago: A search user who is logged in should have the ability to block entire domains from all future results. The benefits of this are many. The cost is very low. Why is this option not already available? Google - we depend on you. Do it."

mistermann 16y ago
>Why is this option not already available? I think the answer to this question is the same reason why google's results have become spammy. Allowing users to exclude specific domains, or even having a "report as content farm" button, are in direct conflict with google's business model, to a certain degree.
- kijinbear 16y ago
  Maybe Google will charge money for the "premium" search service. Take money from advertisers to show their ads, _and_ take money from users to hide the same ads. I'll hate Google when that day comes. At least I take comfort in the fact that programming-related searches now return more StackOverflow links than ExpertSexChange links.
- gwern 16y ago
  I wonder how much is due to user-friendliness. An invisible global blacklist on my search results? How could that possibly go wrong...
  - quanticle 16y ago
    Well, does it have to be an invisible global blacklist? Would it be possible to create personal blacklists and have one for each user? I mean, Google already customizes search results for logged in users, so this wouldn't be too far of a stretch.
    - gwern 16y ago
      > Would it be possible to create personal blacklists and have one for each user? I think that's pretty clearly what we were talking about. There are invisible global blacklists already, as should come as no surprise (even if you haven't run into one of the hits omitted thanks to the DMCA).