4 ms·
In addition to users submitting articles, is there a reason this doesn't have a spider of its own based off something like the Google zeitgeist to seed some top
by j45 8y ago
In addition to users submitting articles, is there a reason this doesn't have a spider of its own based off something like the Google zeitgeist to seed some topics?
This project looks neat, I think first experiences with it would be much more improved if you could seed it with some content.
Maybe this could run my search with other search engines to compare and gain insights.
- misterman0 8y ago"is there a reason this doesn't have a spider" Yes, server capacity. Once I have a better hosting situation I'll start crawling. Thank you, I've tried to be neat this time around. With regards to full-text search, the didyougogo search engine should be able to replace elasticsearch (which is laughable relevance-wise in my eyes) or solr, once the alpha-bugs are gone.
- iovrthoughtthis 8y agoWhy not let volunteers run the crawler on their own machines? Perhaps HN members might offer some spare cpu cycles.
- j45 8y agoMakes sense re:crawling. Maybe a few sites could be crawled and indicate some sample searches that could be run in the meantime.