4 ms·
I don't imagine that I, as a lone person, can actually build an engine to ever compete with Google or Bing, etc. It is not link crawling the entire web. Instea
by shakna 6y ago
I don't imagine that I, as a lone person, can actually build an engine to ever compete with Google or Bing, etc.
It is not link crawling the entire web. Instead it is trying to find information that is current. You'll find relevant and current information from the BBC, CNN, and NPR for example. As well as stuff from places like LowTech magazine, the CCC, and FSF, etc.
I haven't really put any information on the page at all about it. But mostly it scratches my own itch. But I wouldn't see it becoming a competitor if you're looking for "anything & everything", ever.
(Though that 2 hours is actually you watching the nginx cache expire. The database updates hourly.)
- sangupta 6y agoGot it. So it sources data from a whitelisted set of sites and updates every hour. If the curated list can scratch my itch, I would love to come back. You mention you used some sort of NLP (mention of nltk before) - is it for summaries or reducing noise, or for bringing context to search terms.
- shakna 6y agoYeah, pretty much. I'm using nltk for - generating most (not all) of the summaries, and for building most of the tags that get attached to each article. It's also being used when searching the text of a summary.