3 ms·
Kudos for your courage to make your great ambitions public from the start. 1. Does the site do any crawling on its own, or is the public index only fed from su
by tcmb 8y ago
Kudos for your courage to make your great ambitions public from the start.
1. Does the site do any crawling on its own, or is the public index only fed from submissions?
2. It appears Umlaut/Unicode handling needs some work: When I search for "Käse" (German for 'cheese'), I get the response "0 results for 'Käse' in 'www' (0 ms)".
At this point I'm not sure if there's actually 0 results or if it was actually searching for the escaped string.
- misterman0 8y agoThanks! 1. You may submit a page. When I have a little more capacity that just 1 CPU/1 GB RAM I will also crawl. 2. I'll look into it. Thank you.
- kps 8y agoAltavista ran on an AlphaServer 8400¹: maximum 14 × 612MHz CPUs, 28G RAM. ¹ https://en.wikipedia.org/wiki/AlphaServer#Turbolaser_Family https://en.wikipedia.org/wiki/AlphaServer#Turbolaser_Family
- gary__ 8y agoWould the common crawl dataset be useful to you starting out? http://commoncrawl.org http://commoncrawl.org
- misterman0 8y agoYes absolutely. I have been holding off crawling because I have no server capacity yet. That will probably sort itself out pretty soon from the looks of it. When I have the disk space I'll start using their data.