12 ms·
Am I missing something or does this not work for the majority of us who are casual commentators on here?
by dolinsky 17y ago
Am I missing something or does this not work for the majority of us who are casual commentators on here?
- silentbicycle 17y agoIt probably needs more data points.
- riffer 17y agoYeah, there's really two issues here. Most of the techniques for this sort of thing don't work that well for sparse datasets. So given a choice between showing bad results for users with relatively few comments, and not showing any results ... especially when users can search not just for themselves but also for folks they know and enjoy ... Also, scraping the full history of HN is not cool.
- silentbicycle 17y agoA while ago, pg posted an archive of HN comments, specifically so that nobody would have to scrape them. (It was coupled with comments on how the arc server was holding up, IIRC.) I haven't had any luck searching for it, though - there are too many discussions about the ethics of scraping comments, archive file formats, and the like.
- slackenerny 17y agoI remember that too. It was, as you say, a while ago, meaning long time outdated. And I indeed recall it to be pg-posted, but instead all I can find is this non-pg release with all links broken: http://news.ycombinator.com/item?id=173045 http://news.ycombinator.com/item?id=173045. I looked for in all links to tar and zip archives, could've missed something.
- silentbicycle 17y agoYeah, they're 404. Thanks for looking!
- pronoiac 17y agoSee boredguy8's comment near the top for a list of five datasets: http://news.ycombinator.com/item?id=271066 http://news.ycombinator.com/item?id=271066 I'd check the Internet Archive for the files & Google for the filenames.
- silentbicycle 17y agoThat was the post I was looking for. They're 404 now, but thanks for looking.