4 ms·

I was at Inktomi in the late 90s/early 2000s. We were trying to keep up with Google in terms of scale, and we could do everything they could do in terms of link

by diego 16y ago

I was at Inktomi in the late 90s/early 2000s. We were trying to keep up with Google in terms of scale, and we could do everything they could do in terms of link analysis, relevance, etc. Objectively, our results were as good as theirs. They took over in terms of index size because they could add cheap Linux machines easily. We were tied to our expensive Sun hardware, it took us three years to switch to Linux (long story) and then it was too late. tl,dr: relevance doesn't matter if you don't have the result the user wants.

RyanMcGreal 16y ago
Google's build-out has been nothing short of astonishing. I remember when Google indexed the web once a month - and the web was much smaller then. Now my sites get crawled several times a day - and new articles on my news site are in Google News within less than an hour. The mind boggles.
- diego 16y ago
  The main reason their cache was useful back then is that many pages were long gone and the links were 404s. Inktomi marketing always tried to play up our freshness compared to Google. In 1999 they had some pages in their cache that were 3-5 months old. We were pushing new indexes once a week, no page was older than a month. Seems quaint now.
  - nostrademons 16y ago
    I find that the cache is useful now because articles get posted to social news sites, the influx of traffic brings down the site, and Google Cache is a handy mirror that usually seems to have the page already.
- candeira 16y ago
  I just did a search for an error message in a piece of software. The one result from Google was my own pastebinned traceback from five hours earlier. Google indexes random pastebins faster than I can forget I posted them. This still cracks me up, but now I laugh nervously: http://www.ftrain.com/robot_exclusion_protocol.html http://www.ftrain.com/robot_exclusion_protocol.html
jforman 16y ago
Ex-Microsoft here. It's been a while since I worked there, but my recollection is the same: blind, automated testing stripped of UI elements showed that Microsoft's search results were at parity to Google's in terms of relevance. People sometimes forget that all of these large companies have teams of very smart engineers and researchers. Google may be a talent sink, but they're not a talent monopoly.

by diego 16y ago

RyanMcGreal 16y ago

Google's build-out has been nothing short of astonishing. I remember when Google indexed the web once a month - and the web was much smaller then. Now my sites get crawled several times a day - and new articles on my news site are in Google News within less than an hour. The mind boggles.

diego 16y ago
The main reason their cache was useful back then is that many pages were long gone and the links were 404s. Inktomi marketing always tried to play up our freshness compared to Google. In 1999 they had some pages in their cache that were 3-5 months old. We were pushing new indexes once a week, no page was older than a month. Seems quaint now.
- nostrademons 16y ago
  I find that the cache is useful now because articles get posted to social news sites, the influx of traffic brings down the site, and Google Cache is a handy mirror that usually seems to have the page already.
candeira 16y ago
I just did a search for an error message in a piece of software. The one result from Google was my own pastebinned traceback from five hours earlier. Google indexes random pastebins faster than I can forget I posted them. This still cracks me up, but now I laugh nervously: http://www.ftrain.com/robot_exclusion_protocol.html http://www.ftrain.com/robot_exclusion_protocol.html

jforman 16y ago

Ex-Microsoft here. It's been a while since I worked there, but my recollection is the same: blind, automated testing stripped of UI elements showed that Microsoft's search results were at parity to Google's in terms of relevance. People sometimes forget that all of these large companies have teams of very smart engineers and researchers. Google may be a talent sink, but they're not a talent monopoly.