4 ms·

This is inaccurate. When I was running AV operations around 2000, we were running on a couple dozen huge Alpha machines for the index layer and queries. We had

by BryantD 4mo ago

This is inaccurate. When I was running AV operations around 2000, we were running on a couple dozen huge Alpha machines for the index layer and queries. We had a bunch of smaller machines for Web serving, and a high memory set of Alphas as a caching layer. See https://www.usenix.org/legacy/publications/library/proceedings/ana97/summaries/monier.html https://www.usenix.org/legacy/publications/library/proceedin... for the 1997 server count, which was before we got to the three tier architecture. We also spidered constantly. A couple of those huge backend Alphas were dedicated to holding the constant spider index. AV had a well earned reputation for quick discovery, although I think Google wound up faster. We suffered a bit from maintaining separate indexes for the main corpus and recent pages, and I imagine Google handled that better. But the period of time when our main index went to hell was the period of time when we failed to do a new main index crawl for several months. I won’t get into why that happened politically because my memory isn’t perfect and I don’t want to criticize anyone who won’t see this to stand up for themselves, but it’s absolutely the case that we let the index get stale. And I will say that I think the execs were distracted by the idea of challenging Yahoo by buying a shopping site and a local news site of sorts and, unlike the Google of the time, they lacked the wisdom to focus on our primary product. And now I fade back into the hedges, until the next time AV comes up… I suspect a high percentage of my HN comments are on this exact topic. It makes me sad.

bsder 4mo ago
I stand corrected. Thanks for all your work. And I still miss the AltaVista illustrated diagram (Java Applet) that would allow you to drill down and specialize the search results. No modern search has ever matched that, again.
- jhy 4mo ago
  That feature sounds amazing! I tried searching wayback etc but wasn't able to find any more details. Do you happen to know of any screenshots / deeper descriptions for it? Perhaps we could nerd snipe Marginalia Search to add it :)
  - bsder 4mo ago
    I remember it being a Java applet with animations, but my memory could be terribly faulty at this point. Following your comments and trying to trail things out, the only thing I can find is a reference to AltaVista LiveTopics. https://dannysullivan.com/why-search-sucks-you-wont-fix-it-the-way-you-think-203 https://dannysullivan.com/why-search-sucks-you-wont-fix-it-t... I suspect what I used was that "GRAPH" button in the AltaVista Refine picture. We might want to archive that image. It seems like the only evidence that this stuff ever existed.
    - jhy 4mo ago
      Thanks, AltaVista LiveTopics is a good keyword. I found: https://www.researchgate.net/figure/Alta-Vista-LiveTopic-Graph_fig2_247392753 https://www.researchgate.net/figure/Alta-Vista-LiveTopic-Gra... which has some different coverage. I did try to archive the blog you linked but unfortunately the owner is blocking it with robots.txt User-agent: ia_archiver Disallow: /
      - bsder 4mo ago
        That hyperbolic tree looks very close to what my faulty memory recalls. I suspect it was a later incarnation that I used.
- BryantD 4mo ago
  It was a fun place to be. Sorry if I came across as curt — rereading that I was a bit more blunt than I intended. Sick cat has been stressing me.
  - bsder 4mo ago
    No offense was taken. It's always good to hear from the people who actually did stuff to set the record straight. > It was a fun place to be. It was. I was in the hardware side during the Hostile Giveaway(tm) to Intel and Compaq. Good luck with your cat.
BobbyTables2 4mo ago
FWIW, I preferred AltaVista over Google back in the day.
panick21_ 4mo ago
From the oral history interviews, it seem one of the tricks was keeping the core algorithm in cache completely.
- BryantD 4mo ago
  I was not a software engineer but yeah, I think so. Every now and then we’d have to go get Mike Burrows to do some consulting work to rewrite a bit more of the code in assembly.
radicalbyte 4mo ago
Thanks for replying, now I remember why I used to spend all day on this site :-) A lot of the political changes and my exposure to some of the VC people outside of public view have soured tech for me. That and the current cult-like behaviour and clear fraud from the crypto and now AI waves... anyhow, I digress. We were big AV users initially, I think for 2 years? This was 94-97 so my memory of the time periods is fuzzy. When Google came along I have very vivid memory of it providing not only better search results but also faster loads times. I wonder if Google was already geo-distributed at the time? Latency was real then, it wasn't uncommon to hit 350ms (compared to 20-50ms to Europe) and the difference would have been felt back then. It was a killer for Counter Strike.

by BryantD 4mo ago

bsder 4mo ago

I stand corrected. Thanks for all your work. And I still miss the AltaVista illustrated diagram (Java Applet) that would allow you to drill down and specialize the search results. No modern search has ever matched that, again.

jhy 4mo ago
That feature sounds amazing! I tried searching wayback etc but wasn't able to find any more details. Do you happen to know of any screenshots / deeper descriptions for it? Perhaps we could nerd snipe Marginalia Search to add it :)
- bsder 4mo ago
  I remember it being a Java applet with animations, but my memory could be terribly faulty at this point. Following your comments and trying to trail things out, the only thing I can find is a reference to AltaVista LiveTopics. https://dannysullivan.com/why-search-sucks-you-wont-fix-it-the-way-you-think-203 https://dannysullivan.com/why-search-sucks-you-wont-fix-it-t... I suspect what I used was that "GRAPH" button in the AltaVista Refine picture. We might want to archive that image. It seems like the only evidence that this stuff ever existed.
  - jhy 4mo ago
    Thanks, AltaVista LiveTopics is a good keyword. I found: https://www.researchgate.net/figure/Alta-Vista-LiveTopic-Graph_fig2_247392753 https://www.researchgate.net/figure/Alta-Vista-LiveTopic-Gra... which has some different coverage. I did try to archive the blog you linked but unfortunately the owner is blocking it with robots.txt User-agent: ia_archiver Disallow: /
    - bsder 4mo ago
      That hyperbolic tree looks very close to what my faulty memory recalls. I suspect it was a later incarnation that I used.
BryantD 4mo ago
It was a fun place to be. Sorry if I came across as curt — rereading that I was a bit more blunt than I intended. Sick cat has been stressing me.
- bsder 4mo ago
  No offense was taken. It's always good to hear from the people who actually did stuff to set the record straight. > It was a fun place to be. It was. I was in the hardware side during the Hostile Giveaway(tm) to Intel and Compaq. Good luck with your cat.

BobbyTables2 4mo ago

FWIW, I preferred AltaVista over Google back in the day.

panick21_ 4mo ago

From the oral history interviews, it seem one of the tricks was keeping the core algorithm in cache completely.

BryantD 4mo ago
I was not a software engineer but yeah, I think so. Every now and then we’d have to go get Mike Burrows to do some consulting work to rewrite a bit more of the code in assembly.

radicalbyte 4mo ago

Thanks for replying, now I remember why I used to spend all day on this site :-) A lot of the political changes and my exposure to some of the VC people outside of public view have soured tech for me. That and the current cult-like behaviour and clear fraud from the crypto and now AI waves... anyhow, I digress. We were big AV users initially, I think for 2 years? This was 94-97 so my memory of the time periods is fuzzy. When Google came along I have very vivid memory of it providing not only better search results but also faster loads times. I wonder if Google was already geo-distributed at the time? Latency was real then, it wasn't uncommon to hit 350ms (compared to 20-50ms to Europe) and the difference would have been felt back then. It was a killer for Counter Strike.