3 ms·

nice work too :)

• siong1987 18y ago
  How long do your crawl the Ycombinator once? Do you actually crawl every page on YC?
  • xtimesninety 18y ago
    I scrape only the front page once every ~4hrs (minimum) Nearlyfreespeech.net doesn't have cron yet, so I'm using onlinecronjobs.com. If you want to scrape it on demand just go to http://hnweekly.watdahel.com/index/scrape http://hnweekly.watdahel.com/index/scrape
    • siong1987 18y ago
      But, if you only scrape the frontpage, how can you know the points for all the past stories which are not on the frontpage anymore?
      • xtimesninety 18y ago
        It's not 100% accurate, but it's close. The reason I did the website is so I can catch up with the good stories that were submitted earlier during the week (I usually read during weekends).