2 ms·

I found it strange not to mention Altavista. Granted, google ate AV's lunch when they emerged, but as someone who was on the web as it started hitting CS depart

I found it strange not to mention Altavista. Granted, google ate AV's lunch when they emerged, but as someone who was on the web as it started hitting CS departments (even the one where the first web spider was created), AltaVista was the thing that made the idea of a "search engine" actually seem like it was going to be a thing. Sure, sure, pagerank was a huge step up (at the time), and larry & sergey would likely have gone down pretty much that path no matter what. But before Google, there was Altavista: fast, feeling comprehensive and pointing the way into the future.

panick21_ 3y ago
They went to DEC and wanted to join them. DEC was not interested
sillywalk 3y ago
Wow, I forgot about that Altavista (and Hotbot and ...) The Hardware Behind AltaVista [0] AltaVista: AlphaStation 500, 256 MB memory, 6GB disk. AlphaStation 500's handle all external traffic to the site. They run a custom multi-threaded Web server which sends queries to the Web indexer and News indexer. Web Indexer: AlphaServer 8400 5/300, 10 processors, 6 GB memory, 210 GB RAID disk. This model is the most powerful computer built by Digital. These servers run the query engine. The Web index is larger than 40 GB, but most requests take less than a second. Scooter: AlphaServer 4100 5/300, 1.5 GB memory, 30 GB RAID disk. The super-spider runs from this machine. It fetches pages from the Web and sends them to Vista, our primary web indexer. Vista: AlphaServer 4100 5/300, 2 processors, 2GB memory, 180GB RAID disk. This machine indexes Scooter output and serves as a central distribution point for new index data. News Indexer: AlphaServer 600 5/333, 896MB memory, 13 GB disk. This machine keeps an up-to-date index of the news spool: since new articles appear and old articles expire all the time, it is in fact quite busy, even though the index it serves is much smaller than the Web index. News Server: AlphaServer 600 5/333, 896MB memory, 24 GB RAID disks. It maintains a current news spool for the News Indexer. It also serves the articles via http to those of you who don't want to know about news servers but want to read news. " [0] https://groups.google.com/g/comp.unix.tru64/c/aB_z5YXwNMI https://groups.google.com/g/comp.unix.tru64/c/aB_z5YXwNMI
dekhn 3y ago
One of the main instigator of Altavista was Mike Burrows. He left DEC to work for Google in fairly early days of Google. He helped implemented some critical infrastructure including Chubby (a distributed lock manager based on paxos) and always had his hands deep in the "subtle" parts of google3 (threading bits). He's also the 'b' in 'bzip2'- the Burrows-Wheeler algorithm. I'll always remember him, he wore the same engineering outfit every day (light blue button-down shirt) and sat in his one-person at the top of steps of the main building right outside the Ougadougo(?) meeting room so basically everybody walking into the building walked right past him.

panick21_ 3y ago

They went to DEC and wanted to join them. DEC was not interested

sillywalk 3y ago

Wow, I forgot about that Altavista (and Hotbot and ...) The Hardware Behind AltaVista [0] AltaVista: AlphaStation 500, 256 MB memory, 6GB disk. AlphaStation 500's handle all external traffic to the site. They run a custom multi-threaded Web server which sends queries to the Web indexer and News indexer. Web Indexer: AlphaServer 8400 5/300, 10 processors, 6 GB memory, 210 GB RAID disk. This model is the most powerful computer built by Digital. These servers run the query engine. The Web index is larger than 40 GB, but most requests take less than a second. Scooter: AlphaServer 4100 5/300, 1.5 GB memory, 30 GB RAID disk. The super-spider runs from this machine. It fetches pages from the Web and sends them to Vista, our primary web indexer. Vista: AlphaServer 4100 5/300, 2 processors, 2GB memory, 180GB RAID disk. This machine indexes Scooter output and serves as a central distribution point for new index data. News Indexer: AlphaServer 600 5/333, 896MB memory, 13 GB disk. This machine keeps an up-to-date index of the news spool: since new articles appear and old articles expire all the time, it is in fact quite busy, even though the index it serves is much smaller than the Web index. News Server: AlphaServer 600 5/333, 896MB memory, 24 GB RAID disks. It maintains a current news spool for the News Indexer. It also serves the articles via http to those of you who don't want to know about news servers but want to read news. " [0] https://groups.google.com/g/comp.unix.tru64/c/aB_z5YXwNMI https://groups.google.com/g/comp.unix.tru64/c/aB_z5YXwNMI

dekhn 3y ago

One of the main instigator of Altavista was Mike Burrows. He left DEC to work for Google in fairly early days of Google. He helped implemented some critical infrastructure including Chubby (a distributed lock manager based on paxos) and always had his hands deep in the "subtle" parts of google3 (threading bits). He's also the 'b' in 'bzip2'- the Burrows-Wheeler algorithm. I'll always remember him, he wore the same engineering outfit every day (light blue button-down shirt) and sat in his one-person at the top of steps of the main building right outside the Ougadougo(?) meeting room so basically everybody walking into the building walked right past him.